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PREFACE 



This report is one outcome of the Preconference Educational Research 
Training Institutes that were offered by the National Art Education 
Association with the suppoi ‘ and encouragement of the U. S. Office of 
EducatiOii. 

As a beginning it may be helpful to establish a sense of how the 
Institutes were organized, with a brief resume / of the development of 
events that took place. Those who are interested in more detailed 
information will find it in the appendices as indicated. The introduc- 
tion in this report provides a background of information for the content 
that is presented thereafter. 

In the fall of 1969 an announcement of the institutes and a call for par- 
ticipants was issued to all NAEA members through the NAEA News- 
letter, ART EDUCATION, and the NEA Reporter. For those outside 
the membership, direct mail announcements were sent to college and 
university art departments and state departments of education. Since 
the number of applications did not exceed the number that could be 
accommodated, no selection was necessary, and all who applied be- 
came participants. Each Institute had 25-30 participants composed of 
elementary class room teachers, art teachers, consultants, and super- 
visors from the public schools and art education at < ’ \ levels of Higher 
Education. ^Although the training activity was designed to benefit art 
educators who had limited or no background in research, a small 
portion of the participants had advanced degrees and used the program 
to supplement their skills in research in an area they felt an important 
need. 

The Institutes were held during the three days immediately prior to 
the regular Regional Conventions of the National Art Education 
Association in 1970. Approximately eight hours of each day were 
scheduled for instruction and working on problems Large group 
sessions were designated primarily for instruction, but open question 
and answer periods and reports from the small groups also took place 
during this time. The participants were randomly placed into four or 
five small groups for woik sessions. The work sessions started with 
assigned problems and later became a time when participants worked 
on problems of their own choosing. Light assignments for "homework" 
were also given as preparation for the activities of the following day. 

* Appendix B 
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The training activity concentrated on providing the participants with 
a workable concept of descriptive research and the fundamental 
techniques of observation methodology that could be used in their 
respective situations. As a result of this training, the participants 
acquired skill in (1) problem definition and descriptive research, (Z) 
selecting research design for descriptive studies, and (3) the method- 
ology for implementing descriptive study using observation techniques. 

The importance of these Institutes and their influence in the field of 
art education is difficult to assess. The evaluator for the Institutes 
had a dual responsibility. He provided formative evaluation by main- 
taining ongoing commur ication with the participants as to how they 
were responding to the instruction and then feeding this information 
back to the instructional staff each day. He administered pre and 
post test 1 ' on the content of the Institutes and a final subjective Insti- 
tute evaluation that provided the basis for summative evaluation '' 1 . 
However, the true measure of the success of the Institutes will come 
from the increased knowledge contributed to the field from projects 
and influential programs initiated or enhanced by participation in the 
Institutes, Basic practical and theoretical knowledge used or trans- 
mitted as a result of participation can provide a valuable and confirm- 
ing input into the field. 

The main part of this report is in essence a transcription of the 
presentations that were given by the primary instructional staff at the 
Institutes, Some of the material that was accompanied by slides and 
the short presentations that were mainly for instructing and partici- 
pants have been omitted. Unfortunately, the "give and take 11 of shared 
dialogue anch pr ofe s sional interaction, the laughs and the arguments 
that occurred especially in the small groups which brought a sense of 
camaraderie to the Institute s, lose all flavor in transcription. What 
appears .iere is a consolidr.tion of material that was presented in 
shorter segments and has been rearranged into related sections for 
more comprehensive reading. 

No attempt has been made to edit out the individual styles of the con- 
tributors in order to make the text more uniform. Since the manu- 
scripts were prepared mainly from tape recordings of the presentations, 
a conversational quality is often apparent. This quality, combined with 
the color and texture of the J- individual styles, provides a fresh approach 
to subject matter that is usually couched in esoteric language. 
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While this publication may be considered complete in itself, it may 
also serve as an introduction to one of the methods for research in art 
education. For the beginner we hope this may open the door to new 
unde rstandings and activity, and for those trained in research it may 
serve as a companion and desk reference in their professional under- 
takings . 

The planning and presentation of the Institutes and the preparation of 
this publication have required the time and service of the many people 
listed below, and these contributions are most appreciated. A special 
a cknowledgemei. t should be made to Dr. Dale B, Harris and Dr, William 
Rabinowitz who so ably related their areas of expertise to the problems 
of art education. In addition they graciously contributed many hours to 
the consolidation and ■ diting of the trans c riptions of their presentations 
in order to make th> *iblicatiori possible.. Dr, Kenneth Beittel made 
presentations at two 01 the Institutes with slides on his current work on 
drawing using case study technique. His section was especially 
written for this publication. 

INSTITUTE PLANNING COMMITTEE 

Dr, Harlan Hoffa, Chairman, Department of Art Education, Indiana 
Univer sity. 

Dr, Gordon L. Kensler, Project Director, Associate Professor of Art 

Education and Research Associate for the 
Institute for Community Art Studies, Univer- 
sity of Oregon. 

Dr, Stanley Madeja, Director, Aesthetic Education Project, Central 

Midwestern Regional Educational Laboratory, Inc, 
Dr, Edward M^ttiL, Head, Department of Art Education, Pennsylvania 
State Unive r sity , 

N AEA STAFF 

Dr. Charles M, Dorn, Executive Secretary, National Art Education 

As sociation. 

Dr, John Mahlmann, NAEA Project Coordinator, Assistant Executive 
Secretary, National Art Education Association, 



I_N ST 1 TUTE STAFF 

Dr, Gordon L, Kensler, Project Director, Associate ProiBssor of Art 

Education and Research Associate for the 
Institute for Community Art Studies, Univer- 
, sity of Oregon. 

Dr, Dale Harris, Primary Instructor and Research Specialist, Professor, 
Department of Psychology, Pennsylvania State Univer- 
sity, 
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Dr. William Rabinowitz, Evaluator and Instructor, Head, Department 

of Educational Psychology, Pennsylvania 
State University, 

Dr, Kenneth R. Beittel, Instructor Professor of Art Education, 

: v Pennsylvania State University, 

Dr, Marylou Kuhn, Instructor, Professor of Art Education, Florida 
State University, 

Dr, David Templeton, Instructor, Associate Professor, Department 

of Art, Western Washington State College. 

Dr, Brent Wilson, Instructor, Associate Professor of Art and 
Education, University of Iowa, 
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INTRODUCTION 



The National Art Education Association has a growing reputation for 
providing a means of implementing change and innovation in the field. 

In 1967 the national office invited a representative from each of the 
four NAEA regionals along with several other well known art educators 
to attend a planning meeting for the purpose of organizing some train- 
ing institutes for the membership, in 1968 the NAEA in cooperation 
with the Bureau of Research in the U. S, Office of Education, offered 
the first Research Training Institutes in the history of the Association, 
These institutes were offered immediately prior to the regional con- 
ventions and focused on the development of behavioral objectives in art 
for use in research and curriculum construction* Their enthusiastic 
acceptance by the participants prompted the planning committee to 
provide for a continuance of this training with another institute prior to 
the national conference in 1969. 

Late in tire spring of 1969, the planning group met to assess the results 
of the two previous training institutes on behavioral objectives and to 
ascertain desirable directions for future programs. Two decisions 
were made in the early part of the two-day meeting. The committee 
was highly supportive of the work that had been done on behavioral 
objectives, and the reports and evaluations from these institutes offer- 
ed credence to the notion that training institutes were indeed a useful 
technique fox providing a direct input of ideas for effective change. The 
decision to continue offering training institutes was unanimous. The 
second decision required some deliberation, but because of the unlike- 
lihood that more than one kind of institute would be funded, the com- 
mittee decided that the next one should focus on another topic in order 
to reach a different population within the membership, 

The decision to use the topic of descriptive research for the recent 
training institutes came after careful cons ide ration of the needs of the 
field of art education and the establishment of some priorities. The 
outcome was two proposals for different kinds of training institutes that 
would serve different kinds of members. Unfortunately, only the one 
on descriptive research received funding from the U, S. Office of 
Education. 

The purpose of :hese Institutes is easily inferred from the title, and 
one would expect that some kind of training related to descriptive re- 
search was presented. What may remain obscure to some is the 
rationale for having this kind of training; why is it important, and what 
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relevance does it have for the field of art education? The framework 
of information that follows is presented with the notion that without 
understanding the need for descriptive research the importance of the 
content of this institute is not fully recognized. 

Art education in recent years has experienced an intensified interest 
in the area of research. The leadership role in. the organization and 
dissemination of research information have been assumed by the 
National Art Education Association. With the creation of the journal, 
Studies in Art Education, first published in 1959, a systermzed method 
of reporting research to the field was established, 

The monthly journal Art Education periodically deals with research 
topics and is distributed to every member of the organization. Recently 
the Low enfeld Series of Research Mon ograph has been published through 
the national organization, to deal specifically w*ith research information, 
Other research studies and information come from individual institu- 
tions and are less widely distributed. 

One of the current problems in the field is the minimal interest and 
irregular implementation of res*: arch findings by supervisors and class- 
room teachers, Attention to the problem is noted in the introduction to 
the recent publication, Ex emplary Pro grams in Art Education* 

The gap between the theoretician and the art teacher became 
apparent as the programs were reviewed. Many art teachers 
and art administrators feel that theory ar research have 
very little, if any, relationship to the teaching of art within 
the classroom. The skepticism of the techniques of the 
theoretician may be attributed to a lack of understanding of 
such procedures by the majority of art specialists and also 
the lack of dialogue between theorist and art teachers, (10) 

In the art area specifically, these working in the classrooms must 
generate research ideas which are relevant to their needs as teachers 
if pertinent problems are to be identified for further study and the 
overall improvement of instruction. 

The body of knowledge concerned with research in the visual art.- has 
been reviewed by Has tic in the Ye arboo k of the National Ass oc iation 
f or the Study on Art Education , (4) and by Hausman in the Report o f the 
Commissio n on Art Educatio n, (17 ) and it is systematically reported in 
bulletins and journals relating directly to the visual arts and education, 

A number of concentrated efforts on the national level have been made 
to assimilate and correlate this existing research into a meaningful 
body of knowledge, such as, the Seminar in Art Education for Rese arch 
and Cu rriculum Development ,(1) under the direction of Mattil, and the 
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Seminar on Elementary and Secon da ry Sch ool Edu cation in the Visual 
Arts ,(20) directed by Conant. However, the existence of this know- 
ledge and the attempts at correlation to classroom practice are seem- 
ingly not apparent to the art teacher. The lack of awareness of this 
progress in the research area may be a contributing factor to the lack 
of enthusiasm by the art teacher towards research in his field. 

The major problem seems to be that neither the art specialist nor the 
classroom teacher has developed the skills for (1) translating research 
findings into classroom practice, or ( 2 ) initiating and carrying out re- 
search projects that will provide useful information for improving their 
programs . 

In addition, many institutions of higher education are not providing for 
the development of these skills in their programs of teacher training. 

In viewing the present status of art education research in the public 
schools, it is important to recognize that the climate for conducting 
research within the context of the public school is not a positive one. 
Lack of time for research activity, lack of available funds, lack of 
knowledge of research procedures and implications, and lack of inter- 
est on the part of school boards, administrator s, and teachers have 
resulted in a negligable amount of research conducted by the classroom 
ten cher . 



Most researchers would agree that if these conditions exist they would 
hinder the implementation of any type of research activity. However, 
these deficiencies will cease to exist only when the subject area special- 
ist initiates Some enthusiasm for research at the classroom level. 
Specifically in the art area, the art specialist in the public school and 
the junior college must initiate research ideas which would be relevant 
to their problems within the classroom situations. There has been no 
coherent outcry for help from art teachers to study problems which are 
significant to the instruction of art. In fact, the converse might be 
nearer the true picture. That is, funds and opportunities are available 
to t He classroom teacher for conducting research in the visual arts, 
but few are taking advantage of current programs to initiate plans of 
study. 




If art education as a discipline is to conduct a total research program 
which is meaningful to all the individuals in the field, it must begin to 
integrate the classroom teacher into some phase of the program. Most 
research today is initiated and conducted in art education by persons 
directly or indirectly working with some institution of higher learning. 
It might be a better rounded program gaining more popular support if 
some part of the research was initiated solely within the context of the 
school and specifically designed to investigate a problem which lias 
relevance to the teacher and the ii. tructional program, 
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Most educational iesearchers accept the fact that the purpose of 
research in education is a means of expanding subject area and ulti- 
mately improving instruction by application of new found knowledge. 

The role which the researcher has assumed to be important is that 
without various types of .esearch activity to indicate new directions, 
art education could easily become a static discipline. It is evident that 
there are some deficiencies ac pres .nt in the relationship? between the 
researcher and the art teacher as to how the work of one offers poten- 
tially useful contr ibutions to the work of the other. 

Research in art education, as in other fields, encompasses a range of 
inquiries with a variety of purposes and procedures that may be cate- 
gorized under the titles of analytical, experimental, and descriptive. 
Analytic research includes historical and philosophical analyses us well 
as other primarily deductive systems that can be used to derive relation- 
ships that are not necessarily of an empirical nature, Experimental 
Research is devoted to the study of M casual ,f relationships, and is often 
used in evaluating the advantages of one teaching method over another. 
The purpose of Descriptive Research is simple to show conditions as 
'' v exist. Correlation analyses, surveys, case studies, and direct 

ervation are techniques used in descriptive research. (19, pp 35-37). 

I .ce it was neither practical nor possible to do more than a survey of 
the wide field of research during the Institutes, it was decided that the 
focus should be on descriptive research. This focus was selected be- 
cause reported research in this category is often the most easily 
understood and the most applicable to the classroom. It was further 
recognized that because of the nature of the techniques used, the school 
offers a suitable environment for this type of inquiry. 

The need for descriptive research in art education was most clearly 
emphasized by Robert L. I.athrop at the Seminar in Art Education for 
Research and Curriculum Development, He states: 

. . . research in art has been too "spontaneous", attempting to 
cover the "canvas 11 with broad sweeping hypotheses and ex- 
planations before an adequate background of descriptive and 
rational data has been obtained. This is not to be critical of 
the researcher who plows headlong into a highly complex 
area such as teaching and learning. . .It would be unto r Lumt.e , 
however, for any field to have too many "plungers," for un- 
less research in ? >*t education is fundamentally different from 
any other discipline, it is the plodding accumulation of care- 
fully obtained descriptive and predictive data that underlies 
any major theoretical or conceptual breakthrough (1, p. 3 IB). 

As the planning groups and the instructional staff began to work on the 
specifics of the program, it soon became apparent that even the area 
of descriptive research was too broad to cover in any depth. It was 



the intent of the group from the beginning that the Institutes would not 
be simply a series of lectures about research methodologies but that 
the participants must have the opportunity to become actively involved. 
Specifically, it was intended that each participant would write a pro- 
posal that would utilize what he had learned to help solve a real problem 
he had encountered in his current teaching situation. With this in mind, 
the focus of the Institutes narrowed to the methodology of direct obser- 
vation. Observation was selected as the place to scart since it is the 
most fundamental of all the methods and is the foundation of all research. 

It is the hope of those of us who have worked on these Institutes and 
this publication that the contents that follow will activate an interest in 
research and encourage art teachers at all levels to seek more infor- 
mation 1 and try this methodology in their own teaching environment, 
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Observation; The Basis for Teaching and Research 



Dale B. Karris 



I'm making this presentation with two observations in mind. I've 
worked with many art educators over a period of years. I make these 
observations without value judgments, I trust; but they are implicit in 
what I try to do with your profession. First is the observation that 
many art educators distrust science generally; they believe, for one 
reason or another, that it attacks and destroys something valuable in 
your field. Second is the observation that many people in art tend to 
be, in personality, rather who!istic or gestalDsh. They prefer to deal 
with wholes. This attitude consequently leads to a skepticism of or at 
least discomfort with science's most powerful methodologie s - -namely, 
analytical methodologies. What I am trying to do is based on these 
two observations which, as I say, I am not judging at all. The./ are 
general characteristics which we must take into account in any dis- 
cussion of research in art education. 

I am going to avoid technical terminology of research for the most 
part, though not the concepts, I trust. J cannot be entirely innocent of 
some technical terms, for they are necessary to establish meaning. 

But i will try to avoid jargon. Consequently, I will be selective,, par- 
ticularly with regard to observational method. My field is develop- 
mental psychology! particularly child development. Unlike much of 
experimental psyc hology, this field rests its woik heavily on observa- 
tional methods, I suppose this reason prompted your committee' s 
bring me into this picture as possibly offering some help to you. 

Can We Apply Science to Art? 

Byway of introduction, let me attack directly the problem of art vs, 
science with a few comments. 

there arc some who see art and science as quite distinctly different 
approaches to reality. They feel that the systematic rational, and the 
analytic character of science is incompatible with the intuitive and 
experential character of art or the aesthetic, Thereiore, it is really 
quite inappropriate to speak of scientific inquiry into art products or 
processes. Indeed, there arc some who insist that there shouldn't 
ever be an attempt to look at art from the viewpoint of research. 

There is another view, that art and science come very close together 
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at the point of the active discovery or the creative synthesis. The 
artist must integrate his product from a variety of skills and compon- 
ents. So must the scientist, v/ho must reconstitute the phenomenon 
that he has broken down and observed, studies, and manipulated in 
detail. A very interesting book appeared some years ago by Geoffrey 
LaPage, an English medical artist, (14) in which the concluding essay 
put this point strongly and effectively --that art and science really do 
come together. Their goals are almost indistinguishable at the point 
of the creative act or the act of discovery; though their methodologies 
may be quite different at some points, they do have this essential 
aspect in common. It is true that science focuses on behavior and 
dismisses the ineffable. Some artists wish to argue that there is an 
ineffable component in art, something that you can T t observe, that you 
can't scrutinize in any systematic way. 7'his may be true, but it is 
beyond our present science. There does remain a great variety of 
behaviors that can be observed and treated by research technologies. 
Indeed, if we are to t ;ach art, whether its techniques or its appreci- 
ation, we must make some systematic study of these areas. If you 
will grant this point (and I am perfectly willing that you not grant it, 
in which case you'll have to approach all that I say with a good deal of 
reservation) that ther* are some domains of art which are subject to 
systematic observation and treatment, and desirably so if you are 
engaged in teaching, then let's proceed. 

Ob s e rvation - t he Basis of Sc ience 

All science starts with observation. The observation is that something 
occurred, or it did not. One characteristic, and some have said the 
essential characteristic of science, is its public character. What one 
scientist has observed by a particular method of study or procedure, 
another can a ] :o observe by adopting the same method or procedure, 
Individuals with similar techniques and similar concepts will arrive at 
identical cr very close to identical results. This point is essential to 
science . 

The public character of art is not so clearly apparent. In sconce it is 
an absolutely essential ingredient. It may not be so essential in art. 

We have said that art has an exceedingly idiosyncratic or highly individ- 
ual component. We state this poim in various ways, that art is a 
relative matter, that it is what the artist intends, or what he creates, 
or what v.rl is in the eye rd the beholder. Yet, there must be some 
public character to art, or the disciplines of art history and art criti- 
cism could not exist. It seems we must admit some "public character 1 ' 
in art also, even though we may argue, and to some degree success- 
fully, that there is a strong individual component in art- -that art is 
what the artist has created or art is what the beholder conceives it to 
be. 
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Basically, by observation we mean we can supply a description of 
something that has occurred* Someone has said that nothing is so 
durable as a good observation. That is, a well described protocol of 
something that has happened can be examined now, and it can be ex- 
amined two years from now, because it preserves essentially' the 
ingredients of what has happened. A good observation, then, possesses 
the public character we have said was essential to science. We recog- 
nise, of course, that it isn’t always possible to have a full account of 
what happened. As one sits watching and listening to a lecturer, one 
can make notes which reconstitute for him the essence of what the 
lecturer has said. One could, also, if he wished, make some notes 
about the lecturer's habitual gestures, some of his failings in speech. 
Such notes would enlarge the description of the lecturer's behavior. 

One could make some inferences about the state of his health- -that he 
seems to hav^ a cold, but there would be a variety of internal processes 
that would not be available to the observer from his eyeball technique of 
watching- -the lecturer's blood pressure and other physiological indices 
of his state of arousal or emotional disturbance at the moment, his 
involvement with what he has to say, his reasons for accepting the 
lecture assignment, and many other significant items. 

Characteristics of Science and Ari 

One's observations are necessarily selective , partly in terms of what’s 
available to his method of observation, and partly in terms of what one 
chooses to define as relevant to his purpose. The fact that this selec- 
tion occurs is due to large part of one's training by sitting through 
many lectures. One trains himself to listen and take notes, to listen 
to and note the content, and to ignore 01 attend only in passing the jokes 
and trivialities. This selectivity in observation requires that we define 
what we wish to observe, in order to st?.tc whether it aid or did not 
occur. In this case the subject is observational methods, the require- 
ments of good observation, and some of the values of observations. 

The student’s selecti/ity, however, tends to focus his attention on that 
Something, and it screens out or turns off his attention to other some- 
things , 

So it is with observation for scientific purpose. One is concerned with, 
let’s say, the creative aspects of children’s easel paintings in the 
kindergarten, finger paintings in the third grade, and the like. He must 
define what he means by the creative content or the creative expression 
involved and screen out or turn off some other aspects of those paint- 
ings for the time beging, One may decide *hat subject matter and color 
are all that he will attend to. This selectivity disturbs some teachers 
because they want somehow to capture all that’s there. It offends me 
at times when I'm missing something, and I wish 1 could capture it be- 
cause I’m focusing on a particular aspect. Yet I know of no solution-- 





one cannot deal with the whole and deal with it systematically except in 
the most general terms. There is simply too much there. One has to 
do a certain amount of narrowing and focussing. If something escapes 
this process, then regrettably it escapes. It may be a serious short- 
coming of science that it can't deal with all aspects of the whole 
simultaneously. Any yet analytic research procedures have been very 
useful, and so we can't weigh either approach lightly, 

The scientific approach to observation requires that it be systematic. 
One must use orderly, disciplined procedures such that he obtains 
characteristic examples of what he observes. That is, one can 
observe a child in art class, and talk about his interest in his work, 
describing a variety oi behaviors that suggest he is highly interested. 

Or one can talk about a child that he thinks shows a good deal of 
promise in his work. Such discussion may lead to selection of pro- 
ducts and the demonstration of activities that have led you to the con- 
clusion that this is a child with much promise or talent. But what 
res+j rch requires is not just the selection of talented cases or the 
random anecdotes that indicate the child's interest, but the gathering 
of observations by a procedure that guarantees their representative - 
ness. Now this is an essential concept, and I would not avoid a 
technical term. One must show that he has observed a fair sample of 
a larger variety of behaviors, and that he has selected a few, but not 
in such fashion that lie bias his obs ei", ations toward the best that the 
child can do, or the poorest, but a fair sample of his work, One does 
not want just the unusual or the eyecatching. 

The Identification of Criteria 

ing recognized that observation, is selective bui also representative, 
a very important point is that we must identify the criteria by which we 
do the selecting, Furthermore, we must identify the criteria by which 
we judge whether or not something has occurred. We may ask this 
question* Have these children produced works with imaginative quality? 
Or we may choose to speak of aesthetic quality. If we use the words 
imaginative quality, then we must define what we mean by image and 
imagination, and this gets n> into psychological processes. If we study 
aesthetic quality, we must define aesthetic, and we will be involved 
with philosophic as well as psychologic issues. But we must define and 
set up criteria so we know whether or not the particular work that we've 
assembled from these children does or does not contain these qualities. 
If we =i rc going to identify drawings or pointings that contain this 
element as opposed to those that do not, we must identify what they are. 
This identification requires a definition. In science, definitions are 
technical and stated mathematically. And if not stated mathematically, 
they arc stated in words that have limited meaning. One of the reasons 
art educators get nervous when they use scientific procedures is 



because the language of art is often highly idiosyncratic - -even poetic in 
character. It doesn't lend itself to the precision that permits scien- 
tists A, B, and C to identify the same behaviors. This is a very real 
problem, because the language we have to use must shave away the 
highly idiosyncratic meanings and come up with the common meanings. 
In art we find ourselves frequently using similar words but with each 
his own private meanings. 

For those aspects of artistic behavior that are subject to scientific 
behavior, then, we must work toward definitions. While I doubt ‘:hat 
we'll get many definitions that are mathematical in character, wo will 
have to come to some agreement about what we mean. To this problem, 
we do have a sort of answer, and I suggest that this answer can be im- 
portant for us. I try to describe a quality I see in samples of art work, 
and unally I say: ,r I can't tell you, but I can show you an example of 
what I mean, " I shuffle through a pile of paintings and pull one out, 

"Now this has what I am talking about," I say. This procedure of locat - 
ing examples helps when one runs into problems of defining in words 
some component or quality of the work he is trying to identify anc 
obse rve, 

I don't know how many of you know the work that Florence Gooderough 
did in the 1920's and which I took up years later, using children's 
drawings to estimate intellectual maturity, (11 and 12) This scale de- 
pends on the number of ingredients which may or may not occur i:i the 
drawing. In revising her scale I tried to use visual examples of what 
would be acceptable and what would not be acceptable for many points 
on this scale. Sometimes the tests manual gives an example of a 
"marginal pass," Nothing less than the quality exhibited in the visual 
example will pass. 

Another example comes to mind. A psychologist was inU rested in 
studying w r he the r, in fact, psychotics in state hospitals produced paint- 
ings that were different from those made by normals. He listed a long 
series of ways in which such paintings might differ; they included such 
things as color and use of line and mas s, type of b: ush sti ekes, and the 
amount of paint applied in a particular way. I suppose he was influenced 
by his reading about Van Gogh and what had happened in some of his 
work. He proceeded tu set up examples for ail the qualities he wished 
examined. When he prepared a verbal description of the quality “’thick 
paint" as opposed to "thin paint," he gave in actual example of each. 
When he described the distinction between heavy brush strokes and light 
brush strokes, he gave examples, as well as the verbal description. In 
judging a painting, a person could look at examples of specific qualities 
and make a judgment as to whether this or that degree of t lie quality was 
present. He found that giving examples along with his verbal definitions 
improved greatly the agreement of judges with respect to :he qualities 
iudeed. 



Clas sification; the Application o t Criteria 

Having defined our criteria and having aided the definition by examples 
that exhibit it and those that do not, then our procedure is to take a set 
of materials and sort them into groups - -those rhat have t lie qualities 
defined bv the criteria and those that do not have the qualities. Having 
made this fust judgment, which results in sorting materials into two 
piles, we may look through the piles much more carefully to tied more 
precisely the ways in which these two groups differ. This technique 
requires us to refine the criteria and is an essential step in research 
on artistic behavior. 

Usually we say research could go forward in two steps or stages. There 
should be trial work on students who are like the ones you are going to 
study in detail, but not the same subjects, because from your study of 
their drawing you are going back to change some of your definitions and 
criteria. Having got to the point that you are satisfied with your 
criteria, then you apply them to the study group proper. And you ?.pply 
your criteria to all of the subjects that you’ve selected for study. 

The reason why one must separate these two steps is that if one changes 
his criteria in process, he really must start over again to apply the 
changed criteria to a fresh group- Otherwise he contaminates his 
original study by the changes brought about in h s criteria by the exam- 
ination of actual materials. 1 tun’t emphasize this fact too much- -that 
observational techniques cause us to refine and redefine our criteria. 

In my field of child development, colleagues who have tried to define 
their criteria, in terms of theory alone, and then go out to observe 
children's behavior have never been too successful. The simple cri- 
teria defined in terms of one's own ideas and concepts do not reflect 
adequately what occurs in actual behavior. One needs to correct his 
own impressions by actual involvement with material. Try out your 
preliminary criteria and definitions on. some material and then go back 
and correct your criteria in te^ms of what you've actually seen- Having 
arrived at a redefinition, it is equally important to go ahead and check 
out what one has redefined against a new group of subjects, to see 
systematically whether in fact the criteria now work, on this new group. 

The Identification of Variables 

Earlier we pointed out that the first step is to det. nnine if something 
had occurred or not. Having made this distinction, we sort work into 
two piles, those that contain the event and those that do not. Then we 
make more intensive studies *o see wnat similarities and differences 
exist within the two piles of work, especially in the group wherein the 
event has occurred. We seek to extend our original observation and 
make it more precise, so to establish similarities and differences. In 
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distinguishing between drawings that have a creative quality and those 
which do not, we ma> note that th;- creative examples show many 
features which cause us to wish to sub-divide the general class. Thus 
we establish logical categories, and these categories refine our criteria. 

The ascertainable variability in the quality we originally defined may 
take the simple form of amount rather than kind. We fine ourselves 
making a judgment of amount on the general phenomenon that we define. 
We may refine thie quantitative judgment by creating three sub- 
categories - -a just perceptible amount of the quality, considerable of 
the quality, maximal degree of the quality. We may not fully agree on 
specific instances as between two of the three sub- categor ie s, but there 
should be some degree of agreement or we are not judging by the same 
criteria. When we have divided this category into three sub-categories 
of a quantitative character, we have, in effect said that we are dealing 
with a va r iable . Here is another technical term. Something that exists 
in varying degrees, and we have made a first step ir.o quantifying that 
variable by tnis three -fold division. Saying little, some, much, is a 
very crude but nevertheless legitimate kino of measuring instrument or 
scale to place on a variable. As our measurement becomes increasing- 
ly precise, our categories may increase in number. We may discrim- 
inate five or seven amounts. 

Theoretically, we could increase these categories indefinitely and have 
what we face so often in art education- -a > many categories as we have 
examples to begin with! But the reason we are attempting systematic 
observation is to reduce the array of material given us by children into 
a limited .lumber of groups or categories that we can make general 
statements about them. This, of course, is difficult. At professional 
meetings how many papers concerned with children's drawings or art 
quickly come down to showing slides of individual children's drawings! 
These are, of course, interesting, but do not permit us to make any 
classification. We just look at a series of slides. The attempt to 
apply scientific approach to material of this kind requires that we re- 
duce the infinity to a series of fin - categories. Thus, the problem of 
categorization is really significant. That we're doing is reducing what 
was originally a qualitative feature into categories, so that brief 
descriptive statements can be made and summaries can be drawn about 
the phenomenon we're observing. We can't be satisfied by describing the 
quality as infinitely variable. W e can't take 120 drawings and say "Here 
are 120 examples of behavior. " We seek to reduce the variable to a 
series of manipulative categories so that we can make some generali- 
zations about the 120 examples. We reduce our observations by making 
the process selective and by defining e " criteria. Having done that, 
we establish the categories that implement our criteria and give them 
moaning. We then make generalizations in terms of those categories. 
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Oftentimes when we do this reductive process, we fail to retrace our 
steps and to put our results into our original terminology. One can 
illustrate this failure from some of the research articles in art 
education. These studies report statistics, averages, standard devia- 
tions, and correlations, for example. The statistical tables convoy 
information (meaning) in much reduced form. These tables are quite 
meaningful to a person who understands statistics but not very mean- 
ingful to a person who may know a lot about art education but not about 
statistics. It seems to me important to go back from those tables to 
the kinds of verbal description which will help reestablish the richness 
of the original behaviors which these tables merely summarize. 
Statistical tables are shorthand notations of generalizations about the 
observations made under the categories that have been established. 

The Analytic Method 

Essentially, what we are talking about is the analytic method. We 
break a phenomenon down into bits, and classify, treating the parts 
systematically. When we look at parts of art products, the appearance 
is very different from the original material. One feels relieved, some - 
how, if he can show a few examples. The temptation is to return tc an 
exhibition. Examples are excellent. Let the examples, however, be 
selective to illustrate the gene ra lizations achieved by the reductive 
process. Remember that one is always dealing with variability and 
that one selects examples that illustrate certain classes defined on a 
continuum which varies by small amounts. 

Two illustrations of research that have covered this ground may help. 
First, there is Florence Goodenough's scale for evaluating drawings of 
a man from which she derived a judgment of intelligence .( I 1 ) This 
scale illustrates the analytic method. Second, is Professor Rabinowitz's 
work on teacher behavior which illustrates the qualitative approach to 
drawings, making judgments which evaluate the drawing as a whole. (16) 

An Example of a Quantitative Scale 

Florence Goode nough started with certain observations about children's 
drawings. First, she knew that drawings grow more complex with age. 
She also knew that the drawings of brighter as compared with duller 
children show very obvious differences; particularly in the richness of 
detail and proportions of parts. She recognized what must be system- 
atic in her observations. Whe decided she would set a common task for 
children in these terms: "Draw a man, the very best man you can. 

Draw me a whole man; not just the head and shoulders, but a whole 
man." This task immediately restricts the kind of observations she 
could make. She could not tell anything about the child's trea ment of 
scenery, or houses, she couldn't <cll much about his sense of beauty or 
much about his creativity. This r;>'riction lost much that might have 
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been observed, but for her purpose, which was to see how drawing 
relates to the increase in intelligence, it was necessary for her to 
restrict her observations, 

Goodenough's basic assumption was that intelligence grov/s with age, 

Her hypothesis was that because drawings increase in complexity and 
in the use of realistic proportion with age, there rnay be a relationship 
between these two variables such that drawing might be used as an 
index to intelligence. She made the further assumption that the brighter, 
young child is more like the average child who is older than he, and 
that the duller child is more like the average child younger than he-- 
both in intelligence and in drawing. The assumptions in regard to 
intelligence had been well demonstrated by research on intelligence 
tests. The relationship of drawings to intelligence had not been demon- 
strated except in a very general way. 

Dr, Goodenough collected several thousand drawings made by children 
of all grades in elementary schools in New Jersey. She first sorted 
them out by age and went into a big empty room, spreading the drawings 
on the floor. She then walked up and down the arra^ of drawings noting 
down her impressions as to the ways in which the drawings changed with 
age. She got many, many impressions which became the working cate- 
gories into which she then proceeded to classify the drawings. For 
example, she got the impression that, with age, children treated differ- 
ently the shoulder of tire man. Where younger children were not too 
precise about locating the arm on the body, the older children were 
quite particular. Older children were much more likely to include a 
variety of facial detail; young children were satisfied with .he major 
feature s - -eyes, nose and mouth. Older children were mere inclined to 
put on items of clothing. 

Her first criterion, then, became increase with age in the percent of 
chiluren including eacl feature of detail. If there was a regular in- 
crease, with age, in the percentage of children including the feature, 
this item was retained fo^ her scale. If the. item did not show an 
increase, she discarded the item from further study Then she went 
through the drawings and sorted them first by grade then by age within 
grade. Presumably, the children younger thin the typical age for a 
grade were the brighter children who had been pushed ahead. In the 
1920's acceleration was not uncommon. Persumably also children who 
were older than typical for a grade had been held back because of poor 
performance. Presumably th se included many slow or dull children, 
intellectually speaking. This procedure supplied her second criterion, 
that the under-age, presumably brighter children should include the 
item in greater percentages than the over-age cnildren in the same 
grade, If this second criterion was satisfied she kept the item. Some 
fifty-two items satisfied both criteria. Theje items were, you might 



say, fifty -two categories in terms of which drawings could be divided-- 
taken one at a time. She found that if ucicly oc^red the, ui swings 

as a test, counting a plus for each itena in her series of categories 
which appeared in a drawing and counting as zero those which did not 
appear, she got a score --a number of points that varied from child to 
child. She found that this score correlated statistically with other 
measures of intelligence which might take an hour to two hours to give. 
The drawing of man took, generally, about ten minutes. In the element- 
ary school, she believed her hyp Tesis justified. 

Drawings treated in this way gave a measure of intelligence, and this 
measure has proved to be quite a useful, simple measure of general 
intellectual maturity in elementary school children. My revision has 
amply substantiated her hypothesis. She used an analytic procedure to 
identify the component in drawing that relates to intellectual maturity. 
She was not concerned about other features of children's drawing or 
their drawing behavior or ability. She scoreu not on technical or 
artistic excellence, but on clear representation of the idea embodied in 
the item. For example, the item; child draws the eyebrow or the eye- 
lashes, or both. The percent of children who include this point in each 
age group from six through fifteen in representative samples of one 
hundred at each age goes up steadily from 30% at age 6 to near 80% at 
age 13 and then drops slightly at ages 11 and 15, Adolescents, typically, 
become less certain about drawing ability and less willing to attempt 
precise details that small children do boldly, so the percentage curve- 
drops a bit in these older ages. As a measure of intellectual maturity 
this item is no longer functioning s d well at these ages. 

An item which works adequately at the-, oldest ages is the point: feet 

shown in pe r spective --one or both feet in perspective. However at no 
age do very many of the children include this, no more than about 15%. 

In one sense it is not such a good item because it ought to be included 
by perhaps a third of the children to be functioning as a measuring 
device. We kept the item in the scale because we were trying very 
hard to find items that would differentiate among our older children. 

We were nut very successful, and this was one of the few items that 
did, so we kept it. 

Young children frequently draw teeth, \nd so we put that item into the 
scale. Childien at six mid at eight included it, but no one after age 
twelve included the item. Furthermore, while it diift rentiatos the 
bright from the dull at age six in that the bright children arc more 
likely than the dull to include it, at eight and ten it differentiates in 
reverse; t He duller children are more likely to incluo ' it than the 
brighter children. The item, then, v.v s discard* d on two counts --age 
progression, and consistent differentiation between bright and dull. 
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The c c data illu straff* an analytical procedure and how observation of 
drawings can be narrov ed by definition to fifty-two elements scored on 
their presence or absence and treated systematically to yield a measure 
which has found considerable use in psychological clinics throughout 
the country since it was first introduced in the mid \9Z0 ] $. It is a 
widely used test with young children because children like to do it. It 
doesn’t take much time to give or score. It gives a reasonably reliable 
index fur a rough measure. We don't use it when a precise measure of 
intelligence i equired, because making a judgment about a child’s 
school piacen .t or his placement in an institution on the basis of 
intelligence is a serious matter, and we dor^t want to rest the decision 
on any one test, let alone a ten-minute test. We want the judgment of 
a variety cf tests. This is just good psychological prudence. But the 
Goodenough method illustrates some of the things we’ve talked about- - 
the analytical procedure, the discrimination of elements for observa- 
tion, and the definition of those elements. Finally a point I haven't 
mentioned he retofore - -the fact that you can train clerks with no more 
than high school education to score this ce st as competently as highly 
trained psychologists. That we can do this testifies to the public 
character of me phenomenon we have observed. People who follow the 
directions do not require a lot of special knowledge to agree. The 
point is explicit and objective. People can arrive at similar re suits 
after a short degree of training in using the scoring instructions. 

An Example of a Qualitative Scale 




The other research I wanted to mention was a study by Dr. Rabinowitz. 
Pis problem was to identify students in teacher training classes who had 
considerable promise as teachers. His criterion, ultimately, was 
their skill as classroom teachers. He tried to devise a measure, based 
on drawings which these student teachers made, which would predict 
certain qualities of successful teaching. ( 1 6; I'm not going to talk about 
the ability of this test to predict teaching performance but rather the 
construction of the scale, which illustrates ihe development of scales to 
measure qualitative aspects of drawings. He constructed a scale for 
treating the drawings supplied by students, so that he could make this 
other kind of prcdicti\ % Ludy. asked students to draw a teacher 
■.caching a class of children, lie i mpliasi/.cd that this task itaci nut 
require an artis H *^ talent, that In.* could accept sketches however crude, 
so long as they illustrated the student's idea of a classroom. From a 
preliminary scrutiny of drawings he believed that he could identify five 
dimensions or criteria which could be evaluated in the drawings, One 
of these was the relative emphasis given Die teacher in the drawing. 

The second was the initiative expressed by the teacher figure in the 
drawing. The thiid was the psychological distance between the teacher 
and the pupils, the fourth was traditionalism ; n classroom arrangement. 
The final dimension was artistic quality, winch Dr, Rabinowitz put in 
as a control scale t^ make sure that^y^en his judges evaluated tne first 
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four elements they wore not just evaluating some aspect of artistic 
quality in the drivings, ft was iiti^s&ary to demonstrate that when 
judges evaluated the drawings with respect to these categories, they 
%/ere judging something other than just the picas bigness or the skill in 
the students' work from a representation;*! competence point of view. 

It was necessary to define these dimensions somewhat more extensively 
than, by merely naming them. For example. Dr, Rabinowitz said one 
may judge relative teaxher emphasis in terms of the detail of the 
teacher figure as compared to the pupil figure. If the teacher figure 
has more detail, then there is more teacher emphasis than pupil 
emphasis. Also, one ~ould take into account the placement of teacher 
in the drawing; was the teacher figure '‘central* 1 to the drawing? The 
heaviness of line on the teacher drawing as compared to the pupil 
drawing, and the relative size of the teacher figure as compared with 
pupil figures were two addition? 1 criteria by which teacher emphasis 
could be judged* These four criteria were as far as lie got, verbally, 
in hit definition of the dimension, "teacher emphasis," But he found 
he couldn't judge his drawings successfully just with these verbal 
guides alone, so he selected examples which expressed five degrees of 
"relative teacher emphasis, " scaling the examples by a systematic 
judging technique. These examples represented about equal distance 
from "teacher minimally emphasized" to "teacher maximally emphas- 
ized," scaling the examples by a systematic judging technique. These 
examples represented about equal distances from "teacher minimally 
emphasized" to "teacher maximally emphasized,' 1 These examples 
have a qualitative character, but one can lay these five examples in 
front of him and take a pile of drawings on the same subject matter, 
making a judgment on each as to which of the five standard examples 
it most resembles, qualitatively, in terms of the defined dimension. 

The examples supplement the verbal statements of the criteria com- 
prising the dimension. When you have computed an average for your 
sample, you may find that the average drawing by a particular group 
of teachers is 1,7; that doesn't say very much to you. So you go back 
to your scale and say that it lies between drawings numbered 1 and Z on 
tlie scale, closer to l than to 1. This procedure brings your statistical 
shorthand back into the visual language you started with and makes It 
mo iu 'visual" perhaps. 

Actually, when one makes this kind ot evaluation of ,-.u nbf*f rv.itinn, it 
is generally well to judge all drawings on one scale at a time because 
one wishes to keep clearly in mind the dimension and Us criteria. You 
don't wish the judgments on one scale (or dimension) to color the 
judgments on another. Since he used the same judges, Professor 
Rabinowitz allowed several days to go by before judg ng the next dimen- 
sion in order to erase the memory effects of tin yievious judgments. 
Thus he sought tc reduce the "contamination effect 1 ' ir judgments. 
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The psychological distance scale wat defined by two verbal criteria, 
barriers (such as desks, etc.) Intcipu&cd between the teacher and the 
children, the accessibility of the teacher to the children as inferred 
from the gestures represented in the teacher or the children figures 
in the drawing. From an arm-chair point of view one might suppose 
this dimension somewhat harder to judge than the teache r -emphasis 
dimension. But in fact his judges reached a more consistent degree of 
agreement on the psychological distance scale than on any of the others. 
The ’’traditionalism in classroom arrangement” judgment might seem 
to be somewhat easier to judge; yet the judges agreed only to the extent 
expressed by a correlation coefficient of , 53 and was the poorest scale 
from the standpoint of the public character of the judgment required. 

The elements or criteria of this scale included pupil grouping (is the 
class in a single or multiple units), the nature of the seating, the 
reliance on furniture to designate a classroom as such, and the ex- 
istence of special interest areas in the classroom portrayed. 

Many art educators find the quality scale more acceptable than the 
point scale because it preserves the totality of the drawing more 
effectively. Many times as 1 have worked with Goodenough's method, 
the analytic method, 1 have felt frustrated by the tact that there is a 
lot in drawing that I lose by attending only to the defined points. Then 
I have to remind myscif that it is juite all right fur the purpose for 
which the scale was designed. The details were selected to define the 
dimension of intellectual maturity. Of course there are many features 
of the drawing which may tell other tilings about those who draw them, 
and the point scale technique, as defined, misses them, A Quality 
scale can often create the impression that one is retaining more and 
losing less, but again, this depends on what criterion has been selected, 
and how carefully it has been defined. 
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In the Rabinowitz study the artistic quality scale correlated very slight- 
ly with each of the other four; in fact his judges were judging something 
other than artistic quality, and it is important to know this. This 
procedure established, by a kind of elimination process, the fact that 
he was measuring some th: ng . The other four scales inte r cor related 
quite modestly, arguing that they did not merely duplicate each other, 
they were measuring leu^unnuly independent dimensions of whatever it 
was he was measuring. Unfortunately, the research wasn't completed, 
by correlating scores on these scales with measures of teaching pro- 
ficiency achieved in student teaching or, later on, in professional 
careers. Although we do not know what the predictive power of these 
measures was (the purpose for which t hoy were originally intended), 
we do have a clear demonstration that it's possible to construct scales 
which measure certain conceptual dimension of behavior and to do so 
consistently. This is the first step. The second step is to determine 
what the scales do measure. Ihal, of course, is crucial also. But if 

25 



18 



you cannot measure something reliably and consistently it is perhaps 
foolish to proceed further. One musi demonstrate that his definitions 
have a public character. 

Summary 

Thus, the first step in scientific work : .s to determine what you wish to 
observe. This requires setting up some criterion. Usually, this 
procedure first requires a definition and then the identification of ex- 
amples. Having determined which examples illustrate the presence of 
the quality you're observing, and which represent its absence, the 
second step is to determine additional sub-divisions or categories, 
varying in amount or kind. In discriminating such subclasses one 
further defines his criterion, sharpening it, making it more workable, 
Then having demonstrated a workable criterion, carefully defined 
verbally and picto rially, one can give it to someone eloe. If a second 
person can make similar discriminations, one has not perceived some- 
thing worth observing, or he has been unsuccessful in defining it so that 
he can communicate its character to others. One may think that 
artistic merit can be judged, but if someone else similar in background 
and orientation cannot use a definition of artistic merit to reach reason- 
ably similar results, then something is wrong from the point of view of 
science because science insists on the public charactor of information, 
on the replicability of results* 
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Varieties of Observation 



Dale B, Harris 



la the previous section we started with the notion of something we could 
observe. Wc asked the has A question- -has something happened or has 
it not? Observed in this way we sometimes speak of a characteristic 
as an attr ibute - -it exists or it doesn't. We may divide people into two 
groups according to an attribute - -th ose who possess a quality and those 
who do not, A person has curly hair or straight hair. We can, some- 
time, break down attributes into classes based on more than just 
presence or absence of one quality. We may decide that "curly 11 hair 
breaks down into three sub -clas scs - -wavy, curly, and kinKy, Another 
example, one might take a pile of children's drawings (when they have 
been free to select their own subject) and create a classification as an 
extension of the attribute system, according to the subject matter 
portrayed; landscapes, human figures, machines (automobiles and 
vehicles of various kinds), etc. Such a classification is sometimes 
called a nommal scale; one classifies objects into a discontinuous 
series or categories by naming. For a nominal senile there is no in- 
trinsic.. or necessary arrangement or order to the categories that com- 
prise it. 

However, we may go a bit further. The minute we sa^ that something 
exists, wo can very often say that it exists in some amount. The mom- 
ent we try to define "amount , 11 we're talking of a variable. An attribute 
which can take any one of a number of quantitative values is a variable. 

In such case, we set up a set of categories by amount, those that have 
none, one, two, three, four, or more of this characteristic or , in more 
general terms, none, some, considerably, very much, and extreme 
degrees of the quality. As in the example of an attribute which can be 
classified nominally, so we may classify into categories of amount, 
either counted or estimated. But now we can see a sensible order to our 
categories, from least to most. We now speak of an ordinal scab:. 

Nominal classes can be arranged in any way we want without destroying 
the characteristic that v.e'ro classifying. Ordinal classes can be arrang- 
ed in only one way, or we destroy something of the i. ha ra cte ristic that 
we're classifying. An ordinal scale for variables can take, and often docs, 
take two forms - -qualitative and quantitative. In the quality scale the cate- 
gories are arranged in order, but the steps arc defined by examples. The 
intervening steps or degrees are not estimated; therefore the scale, 




0 



ERIC 



zo 



though ordinal, remains qualitative. The amount aspect of the scale 
is reached by estimating. A quantitative scale is illustrated by the 
Goodenough Drawing Test. (11) The number of points scored on it may 
be counted to yield a quantitative measure. The scale is ordered from 
0 to 52 points. One may group these units into classes of several 
units (3. or \ or 5) and reduce the number of groups into which one 
would sort his results. But technically the scale runs from 0 to 52. 

One must distinguish, then, between the quality scale in which one 
nakes a judgment and the quantity sr a'e in which he counts or measures 
with a quantitative scale. 



The Idea of Dimension 



Fundamentally, when we construct scales we are dealing with dimen- 
sions. A dimension is some quality or aspect or property of an 
experience or of an object which can 1 e defined and which takes 
different values or degrees. W hen we discriminate among these values 
or degrees, we define a contintum. This continuum we see as a vari- 
able. Furthermore, this continuum or variable must be pure, or 
unidimensional, in the sense th it it deals with only one clear-cut 
feature of the quality that we are dealing with. When one takes a col- 
lection of children’s paintings and asks himself how he may describe 
these, lie must first determine possible dimensions in terms oi which 
he might describe the paintings. Examples might be, subject matter, 
color, clarity or ''muddincss" c f colors, quality of line, use of space, 
use of mass, etc. One art group identified more than fifty features 
which might be considered as dimensions of drawings. Now, that is a 
lot of dimensions to work with, yet everyone of them could, theoreti- 
cally* yield a continuum along which qualitative or quantitative 
categories or classes could be defined. Taken altogether these dimen- 
sions would describe all features of the set of drawings that this group 
examined. 



Thus a dimension is equivalent io a continuum, to a variable, and this 
dimension is defined by the ope common property that runs throughout 
all example; s or degrees. If you mix two or three properties into your 
definition of a dimension, then you're in trouble. For one thing, the 
reliability of your judgment goes down. You seek to keep one property 
well dt 1 nod, with the gradations or the categories e s tabl i s-hed uni- 
dimen. -nally, aiul then you devt lop as many different dimensions as you 
need to adequately describe tin 1 material you're working with. 



Dimensionality has one other fea ure which should come into our think- 
ing. Dimensions may exist at various levels of abstraction, from very 
specific, and concrr-te, levels to very abstract levels. You may have a 
dimension mu It rt s amount of pressure exerted on the pencil; this is 
very cum n to. Vmi may have a donension which speaks of the 
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character of the space enclosed by the line, which you tl en describe 
in some way. Such a dimension is at a more abstract level. It’s 
dealing with a broader kind of phenomenon than just pencil pressure. 
When you get up such concepts as aesthetic quality, you're dealing 
with even a broader feature of the material. 

WImi dimensions vary from the very concrete to the very abstract, we 
can often see a kind of order among them. We may then speak of a 
hierarchy of dimensions or of meanings. For example, you can have a 
set of categories which you call fruit. This concept creates a set of 
disparate categories which constitute a nominal scale because these 
are apples, and pears, and plums, etc. --all distinctive, separate 
kinds of iruit. Similarly, you may identify many categories of veget- 
ables. Fruits and vegetables have certain features in common- -both 
are perishable plant products, seasonal in character. Both arc 
edible and found near each other in markets* Each is a class term and 
in a sense they are coordinate. These and many other items arrange 
themselves into broader and broader categories, until we speak of 
11 food" - -a general term which embraces a hie rare] y of terms. A 
hicraichy is built from concrete to more remote levels of abstract 
concepts, D imens ion s have thi.; properly of arranging themselves 
along degrees of ubst i\i ct mn into a hierarchy. This iValuie of dimen- 
sion H v eil to keep in mind when you l n; building quality' scales. You 
may be able to arrange the dimensions you identify into a hierarchy. If 
so, you have, taken a significant step toward establishing i . tea n ingJul 
relations among them. 

To s amma r i &c , one fundamental aspect of rcsearcl is the defining of 
dimensions, and establishing categories within thei \ to permit one to 
classify observations m appropriate ways, so that general statements 
can be. made. This is, one* seeks to reduce* the total number of classe s 
t rum N - tin number ut subjects or examples of work you're working 
with, to a much more 'united number of groups, Eich group or class 
will exhibit some intrinsic characteristic that holds it together, that 
makes it a class, that defines it, and at the same time, permits you to 
observe the regularities or continuities among these classes. From 
tin,- jo-mess i.p f 1 u * si fit. a t inn m observid ph monii n i 'you < an make 
i i'll tin l’ e j u* r. i 1 i / tl mn s a bn Lit the m. i U r i,i ! . T hat is, ymu ( ,m d tvde yeur 
ihibHeJi’s br.tv. ine* by «g< nf (J rtkl ,nul by subject mitU-r. Hum you 
cun Jim u u’fii" i -i 1 t ei iitji t * .^ik h as ’’buys arc mo*e likely* to do ships 
than girl*”; ’ prim. try children art more likely to draw flowers and 
bird* than i rite J'mediate children, M etc . Or you may work at a more 
abstract level a juI define dimension* which show use of line - -whethe r 
and hn \v lines enclose space, etc. V u;i then say children at the primary 
level are unlikely' to make abstract forms spontaneously, or they're 
very unlikely to use perspective. One* makes such general statements 
after lie has defined the property he i looking for, sorted his materials 
into the categories, and made his estimates or his counts. 



Identifying Meaningful Dimensions 



The kind of dimensions and the categories that you define within these 
dimensions will depend on the scope of the task you r ve initially given 
your subjects. Again, I’m using as illustration the drawing area. If 
your instructions allow children to draw anything they like, you'll need 
a much broader set of categories than if you say '’draw a man, IT be- 
cause you can get so many different kinds of drawings* To make 
dimensions manageable in your research, you may have to set nar ■ 
rower limits thant "draw anything you choose,” Your task is consider- 
ably more manageable if you say "draw a picture of your school yard 
at recess time.' Here you've limited the subject matter, so the 
number of categories you must establish for classification becomes 
specific and thus more manageable. 

The question should be raised: "How do you know what categories arc 
going to be important?" "What dimensions, among all possible ones, 
should be studied?" Material may be treated in an infinite number of 
ways. What you will observe, and dhnensionalize, will depend on your 
problem, your question, and on your previous experience and your 
knowledge of previous research Sometimes the selection of fruitful 
dimensions depends on hunch. Scientists in many fields use this more 
often than they admit. We sometimes call this factor "intuition," and 
psychologists are sometimes inclined to toss it aside as mystical. Yet, 
I'm convinced of two things: first, intuition is net completely divorced 

from training. A person who knows a field very well, is going to 
function intuitively more adequately than one who doesn't know that 
field very well. If you're going to do research with children’s drawings, 
you’d better have a lot of experience with children's drawings. If you 
do, you will select dimensions more knowledgeably than if you come at 
them cold. If you've handled lots of children's drawings, looked at 
lots of them, you'll have a better informed and successful intuitioi in 
regard to the kinds oi dimensions you set up to classify them on. 

The other thing 1 would say is that people do differ in this intuitive 
capacity, and beyond that 1 can't jay more. PI veil the informed and 
experienced people differ; sorn»* pick fruitful hunches more often than 
others, This feature of intellect remains something of a mystery, We 
don't understand it nearly as well as we understand problem solving. 

The General Observation 

There are a number of ways observations can be made. One of these is 
certainly direct inspection and description. JT1 just call f his "eyeball- 
ing." One of the time-honored techniques in child study has been the 
running description of behavior from watching what happens. Much can 
be learned from general observa. usually that behavior is much 
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more comples ind difficult to describe than one assumed. This is an 
important lesion to learn. Occasionally I get experienced teachers 
taking my child psychology course. I send them out to observe and 
record everything a child does in twenty minutes of free play. They 
often object to this, and say, Ir but I understand children; I've taught for 
ten years.' 1 After a certain amount ot protest, they'll react that "I 
did'nt know children could do so much, particularly young children. M 
"I found myself hard-pressed to put down everything that went on in 
twenty minutes even in general terms." That's one discovery from 
general observation. 

Next I ask that the observers do one other thing - -sepa rate what happen- 
ed, the behavior, from their own inferences and interpretations, which 
are to be noted in the margin of the behavioral observation record. The 
student writes "The nursery school teache r took the child by the hand 
and led him, unwillingly, over to the sandpile." And I say, "Well now, 
wait a minute --the teacher took the child by the hand and led him to the 
sandpile, Unwillingly was your inference. What led you to say unwill- 
ingly?" "Well, the child cried a little bit, or held back," "Well, that's 
behavior, write out that observation as behavior. The unwilling is your 
interpretation and should be noted in the margin as an interpretation of 
behavior observed," And we go through this process over and over --it 
takes a bit of doing --to separate the inferences from the behavior. 

By the time you get people separating inference from behavior reason- 
ably well, you'll find thal they make a couple of discoveries, that it's 
very difficult to make strict behavioral accounts and get everything 
down, because behavior is very rich. Very often students will say, 

"Why don't you just send me out with a movie camera? It would be 
more successful than my trying to write all these notes, " The second 
discovery is that when all the interpretations are sorted out, behavior 
descriptions become very dull. Indeed the interpretations, the dc scrip - 
tive s - -"big , " and "little," etc. , adverbs "willingly," "happily," etc, 
make the behavior descriptions interesting. When you pull them out and 
put them among the interpretations, the behavior descriptions become 
Vc r y du 11 , 



A very extern- ivc £ct of researches on the observational method, done 
by Roger Barker at the University of Kansas, (5 and 6} has deliberately 
reintroduced the adverbs into running accounts of child behavior be- 
cause he finds that straight behavioristic records don T t give him very 
usefal research material, psychologically speaking. You can't observe 
a child's intentions or goals, you infer them. And the minute you pull 
goal directed behavior out of your observations, your observations be- 
come fairly pointless, 



Observing the Stream of IKhaviur 



o 





Barker built his whole set of observations around sequences of behav- 
ior, which he separated into episodes whenever the child's goal or 
action changed. Bill and John were playing in the sandlot with some 
old crates. The behavior record goes on to make many observations 
relating to their handling of material, skill, and problems they met 
and solved, etc. The boys were building a shack, but they dropped 
this behavior, got on their bikes and rode down the street, At this 
point their inferred goal changes. They went down to the drugstore for 
some ice cream. First the goal was to construct a hut; now the goal is 
to get ice creams. This change of behavior represents a way to 
unitize the behavior. Barker's whole object was to treat the stream of 
behavior as a stream, and yet, break it up into units that could be 
handled psychologically --categories, it you like. He found that he could 
do this by inferring the cliiid's purpose; indeed, he had to infer purpose 
to study behavior in context meaningfully. lie could evaluate these 
units of behavio.. in a great variety of ways. For example, he could 
evaluate a child's behavior in terms of the apparent clarity of the goal 
set for the child when tasks were assigned him. He could evaluate the 
amount of frustration experienced in sequences of episodes, lie could 
sort episodes into social and non-social, and study them in new ways. 
Many interesting facts came to light when he studied behavior episodes 
across many children and across many episodes. He found fc r exunple, 
that the clearest goals for children are set, not by teachers, not by 
parents, but other children. Teachers actually are leat clear in making 
assignments and setting goals for children in the elementary schools. 
Parents are a little better in setting clear goals than teachers, but not 
much, 




One may get the iinpressio. from some of the clinical literature about 
children that their lives art* full of disappointment s and frustrations, 
being put down by adults, etc. Barker found that if you really take the 
ongoing stream of behavior as it occurs in school, on the playground, at 
home, etc, , break it into episodes, you may evaluate them against a 
variety of dimensions by fitting tlu* behavior ..gainst rating scales, one 
scale to each dimension. By this method, Barkei murid that a child's 
rough experiences are very minimal. Acute frustrations occur inf re - 
quently, A lot of a child's experience is kind of pointless, but there is 
much mure constructive than destructive in tlu child 1 « daily routine. 
Adults, teachers, end children ofier inure help than hinderauc* . Barker 
was able to also o*kc the school episodes, along will; tin \ithe v episodes, 
and classify them according to the leurmng significance of tlu oi episodes 
as they occur --did an episode seem to have a significant impact on the 
child, change his bcdiavior? Yfe find that more learning occur * uut of 
school by far than occurs in school, and that significant learning 
episodes ir. school per child may occur as seldom as once- in forty 
minutes. Mostly, the child is just sit ing there. Well, we've known 
this in a genera! way, and we liavt 't been very happy to face it. Data 
like these bring one up short. 
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Making Judgments in General Observation 

The straight behavior] stic, observational account is, then, not neces- 
sarily the most useful. One may learn a lot about observation by 
forcing himself to record only behavioral descriptions. However, 
subjective or experiential aspects may also be observed. Such obser- 
vations often upset some of our cherished pr econceptions , Such data 
give us quite a different picture of the child's life from what our 
preconceptions would have us believe. These running accounts that 
Ihn talking about can be done in free situations or in controlled settings. 
You can do them, for example, by giving children certain materials to 
work with and observing what they do with these materials. Or you may 
give them access to a great variety of material and see what they 
select. You may give a specific assignment, saying "accomplish this.'' 
Immediately, depending whether you've set up a very free or controlled 
situation, you require different kinds of categories. Once you have 
gotten the materials together, you classify them on appropriate d: en- 

sions or variables, assigning them to categories within each va de 
01 * dimension, In other words, you scale them. You may have 5-1 
dimensions that you’ve identified as possible ways of evaluating a 
material, but you may find that in fact you have only 8 or 10, r ihis is 
a common experience, I T ve heard Professor Hoiltcl comment on this 
fact frequently, that one can think of many more ways to handle a 
material than you actually, psychologically, can use, Se^le will be 
into rcorrelated, sonic of them intercor related sc highly that there is 
no sense in making separate judgments. Although the scales may 
utilise different words they in fact require essentially the same kind of 
judgment, This problem runs all through the descriptive terms that 
you use in evaluating art products. Very often our meanings overlap-- 
we think they’re distinct, but when we come to make the several judg- 
ments, we find we're judging fundamentally the same thing. And so in 
research we have to reduce the number of scales wo apply to the dis- 
tinctly diiie rent judgments that we’re operationally mailing. 

The Sele ctive Observation 



Wi have disius-u d the mnning account of behavior* There is -:Po the 
ob‘o rvatiun u: behavior in a schcled situation. These situations a n* 
selected a c c ordin j to a dimension- -.in activity or product uimen Hun, 

The obsn rvation is limited in time ami the kind of situation you have 
selected in which to observe behavior. The resulting record can he sub- 
jected to rating. This observation also can be a training device--to 
make one sensitive to the distinctions between judgment and infcreiuv 
and wh it actually occurred, 



l or research one is much more lik Ty to use selected situations than 
the running *■ r count of behavior. How i vu, the general running account 
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method often can be helpful in defining your problem and determining 
what specific observations v/iiJ be useful. For example, if your 
problem is the general one of how may \vc free -up children's drawings, 
the best thing to do is to watcli children draw for awhile. Then as you 
define your problem more precisely you can determine the observations 
that you can treat more systematically. You select some drawings 
which will help you establish categories expressing degrees of freedom 
or constraint. Then you begin to think of what you can do as a teacher 
to inc r ease or to decrease this dimension. 

At some point you may wish to include in your observations a recorded 
statement by the subject. Now this gets us into interviews and question- 
naires. The subject reports on his own behavior as directed or struc- 
tured by the questions put to him --eigther written or oral questions. 

This type of observation is useful for aspects of behavior which are not 
easily' observable; for example, a child's preference for one kind of 
medium as contrasted with another. You could get some idea of pre- 
ference observing the child’s behavior in spontaneous situations over a 
long period of time, or in free choice situations where, for example, 
you might note that lit: usually selects crayons over finger paints, But 
wliy not ask the child directly what he'd prefer? This is a legitimate 
approach. You can get choice information economically simply by ask- 
ing. 

A questionnaire or interview requires careful preparation in phrasing 
questions which will elicit the observation and interim tion you want. 

This is harder to do than you may think. No one should ever sit down 
and write out a questionnaire or interview and then go out and gather 
data. One always must pretest his questions -- sometime s s eral times. 
A good example is the Eisner's Art Interest Scale, (7) This is i scnedule 
of questions which can be used to collect information about interest in 
art or previous experience with art. 

This direct approach, too, is important in appreciation studies and in 
process studies, What you're getting at is the inner psychological 
processes that .ire not available to direct observation. You can say, 

’ Well, technically, maybe they're not * k ven available to the subject him- 
self; maybe these .ii'u unconscious phenomena, and Ju cv.n't tell you in 
words what's going on," 1 his is a possibility, V.ore are undoubtedly 
some aspects uf appreciation that « re not available to a subject con- 
sciously, h-laybi you'll h.ivt* to be paitunlnrly clever in d^visin^ some 
indirect methods of observation. Some psychologists have been particu- 
larly ingenious at getting indire* t measures. Hut increasing!/, as yciir 
measure become- more indi’iet, you have more and mom difficulty 
defending its validity, 1 be quoth uiiiiie, ov cheek list, oi similar 
device is most legitim tr for those arc as of experience in which you 
have reason to believe a person can make some judgment on his own 
behavior. 
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Tests and Work Samples 



Then there are tests or work samples. I'm talking about art products, 
the products of art behaviors; I'm not talking about personality tests, 

I am talking about tests of skills in art, or work samples of skills in 
art. Give a person some material and see how he works with it. How 
does he handle his brush with oils, with watercolor? How does he 
handle clay? If you set up a test or work sample, you assume thaL 
there's a body of skills that you can sample systematically. You ask 
a person to perform the tasks that you've sampled, and record his 
observed performance on each skill or operation. You accumulate 
these into sums of scores or possibly a profile. What you're really 
getting at here is observation of technical proficiency. Probably this 
appro*, ch would be used in art school much more than in public school 
art education \vh„ re, as I understand it, your emphasis is less on 
technical proficiency than on other purposes. But, depending on your 
educational objectives, if you were trying to embrace or develop 
technical skill with a particular medium in the sixth grade, it would 
be perfectly legitmate to build a set of tests that would permit you to 
make ooser vat ions about the child's technical skill, Finally, we should 
note that in all research the descriptive task is required at some piont. 
In constructing standardized tests, a great deal of observation goes 
into the work. Someone may say, "I r m going to use EllioL Eisner's 
Art Interest TesL with a group of high school freshmen, and divide 
them into those with high interest and low interest and then study their 
behavior in drawing and sculpting,” Well, the direct observation you 
might make with respect to art interests has already been done for you 
by Professor Eisner. In constructing his test he put into it a great 
deal of direct observational work which you accept as an accomplished 
fact. You take the test as a developed instrument and begin your 
observation at another point llav >g used the test and sorted subject 
in*o two groups, then you observe their behavior in art class, You 
take the test for granted, I merely make the point that you can'L 
undertake any research without embodying observation. When you 
give tests, you're accepting somebody vise's observath us. 



The Case Study 



Dale B. Harris and Kenneth R. 13 c it t e 1 



A. The Case Method i.i Art Education 
Dale B. Harris 



There is a particular use of the observational method which has con- 
siderable application in art. The case study should have considerable 
appeal to art educators because there is a minimum of quantitative 
analysis involved. It is globalistic, and permits the synthesis of 
materials in general statements about one person, or one production. 
Those artists who dislike breaking a phenomenon into elements or in- 
to numbers rnay feel they stay closer to the significant material by 
using the case method. Many art educators deal with groups of child- 
run, and they may believe that the case method is not practical for 
them. However, it may be useful 'or particular studies, even when 
one is concerned with groups. The group process itself can be made 
the subject of case study. 

Broadly considered, a case stud/ consists of the collection of a wide 
variety of data and information about a single person, grovp, event, or 
process. One seeks to put together a complete picture, from which he 
may gain greater understanding of that person. Frequently, cases 
are studied collectively. That is, one gathers a scries of cases, col- 
lecting similar data on all of them. He them reviews the cases to 
arrive at some general statement formulated from the consistencies 
or trends he finds among all of them. 

The Case Method Gives Particular Information 

If one wishes to describe the status ui a group of children from a 
collection of individual case studies, lie is not likely to gel good infor- 
mation by this procedure. That is, urn- cannot get information con- 
cerning paiametcrs or norms. One can't icully tell what any 
population of children is like, because one rarely selects cases to 
rep re sent a population* For example, in ihe examph Professor 
Bcittcl presents, he advertised on the student bulletin board for sub- 
jects who would be Willing to draw for an h r a week over a period of 
ten weeks. He had no control ovt - the selection of subjects except 
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among those who came in response to his advertisement. He got some 
very interesting people, including a math major, a science major, an 
English major, and so on, but it's clear that this group did not consti- 
tute a random or even i opr es cntative selection of Penn State students, 
They were students who were curious about or interested in drawing., 
or they wouldn't turn up as volunteers. Such cases are self selected; 
they are not a random set. Such students give useful research infor- 
mation for Professor Beittel's purpose --to study p rocess in drawing. 
But they do not give a picture of what drawing behavior by the typica l 
Penn State student is like* 



Nevertheless, by making such intensive cztse studies, of drawing 
process, the investigator can get a very clear picture of how individ- 
uals go about working --the strategies and the imagery they utilize --by 
getting thoroughly acquainted with these individuals in depth. For this 
purpose, except for skilled artists, almost any student will do. Sev- 
eral case studies will give Professor Beittel some idea about variabil- 
ity and constancy in process, "thus, for process information, for 
knowledge of a person in depth, for forming hypotheses about how 
something is accomplished, the case study is a powerful exploratory 
device . 



One may make applications of the method other than that described by 
Professor Beittel , He can collect samples of artwork of a particular 
individual over a period of time* One may keep notes on the comments 
made by one child artist while he works, notes concerning his apparent 
motivation or his method of work or the like. One may keep notes on 
classroom behavior t ha t he feels throws light on one particular child 1 s 
interest, personality, and the like. He may ask the child to write an 
essay or to make a statement about his interest in and his previous 
experiences with art; he may ask the child to write about a particular 
project lie lias done, One can economically collect information regard- 
in'.' prior art experience, by an inquiry sheet, to hr included with the 
case null a. 

A teacher can kec; uoU concerning a significant art experience he 
introduce^ in class; ho may, tor example, introduce some intense 
sensory experience, moving music, or tactual exploration of a variety 
of textures and surfaces, and then ask the children to draw or paint or 
model, Hl nny keep note s ofwli.it children say and do, as well as 
their resulting products. Thus one* may make r, case studies'’ of groups 
or of individuals. 



A Study of Drawing Process 



Other Uses of the Case Method 
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What ar : j the purposes for which one might accumulate such data? One 
might be interested in accumulating records of individuals that he feels 
are particularly c reative - -imaginative people in an artistic sense. You 
just want to accumulate these for a time to sec what you get. Tn tins 
way I've been keeping notes rather casually on some exceptionally 
gifted children I run into from time tu time. I've not done it particular- 
ly for research; I do it for teaching purposes, to have some specific 
illustrations for lectures. 

You may wish to stud; individuals who have a particularly strong 
imagery or a particular kind of imagery. If you follow the late 
Professor Low enf eld's tradition, you might want to study haptically 
inclined individuals, because such persons do not come along frequent- 
ly in our visually oriented culture. You would keep notes on such 
individuals until you’ve built up a series oi cases giving you a better 
understanding of this dimension of perception. 



One of the best uses of the case study is simply to collect longitudinal 
records or samples, Wc have relatively few collections of such. 

Helga Eng has published two books on her niece's work, (8 and 9) 

Harold McCurdy, a psychologist in North Carolina, made a study of 
the drawings of a man (13) by a boy over a period of about fiv years. 
There's a published monograph concerning the work of a boy who was 
particularly interested in drawing trains, (13) A former student of mine 
in Japan has a remarkable collection of the art work of his four young 
children. He has kept everything that they have constructed, painted 
or drawn, including full notes of what they said at the time. He has an 
enormous pile of material which I have begged him to photograph in 
order to preserve, as this kind of collection is most unusual. Child- 
ren's products are seldom saved systematically. 



All these are case studies ami may help give an understanding of 
artistic processes when we have u store of information in depth. I 
have some data collected in South America from children who had 
never used paper and pencil. Indeed they had never seen pictures. 
These children Jived in a remute, isolated area, high in the Andes. 

The two can’s presented here are by a four -yea r -old boy and his five- 
year-old -brother, 1 had these boys draw daily for about seven weeks, 
The samples included here were taken at intervals from . ne entire 
series. The work of the full r -yea r -old is particularly interesting be- 
cause the collection includes the transition from scribble to repre- 
sentative drawing, We showed him v. hat a pencil would do and asked 
him to til 1 the page with pencil v.a *k, and this is v. hat hr did (Kig. 1). 



Studies of Growth in Interest or Skill 



An Example of a Longitudinal Study 
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The spiral scribble was our demonstration of what a pencil would do. 
Thereafter, our instruction was always to draw an object, generally a 
man. Fig, 2 shows his work two days later. Five days after the 
initial drawing we do get a rudimentary man, along with scribbles 
(Fig, 3}. Two days after that the scribbling is much reduced (Fig, 4), 
but shortly reappears (Fig, 5), and is still present two weeks after the 
initial drawing (Fig, 6), After another week v/e have several forms 
(Fig. 7) with minimal scribbling, Note that the transition from scribble 
to figure drawing did not come all at once, nor is there much evidence 
of a progressive exploration of various forms of scribbling. 

The following September, I visited the village again. Here is the first 
drawing (Fig. 8) that this little fellow produced, after six months of no 
drawing. The following day we get a much better form (Fig, 9). His 
five -year -old brother also shows a remarkable progression but' with- 
out the definite transition from scribble. Fig 10 reproduces his very 
first pencil work, Again, the spiral figure at the top constituted our 
demonstration of the pencil, The following day there is a crude "man," 
along with some scribble efforts (Fig. 11). A week later all scribbles 
have disappeared in favor of forms (Fig. 12), Figs. 13 througl 15 were 
selected at weekly intervals from the total series, produced daily. Fig, 
16 presents this boy's first man after six months of no practice, and 
Fig, 17 was pr oduc v d a w e e k la te r , 

From such case studies I can't get statistics, but one does get some 
appreciation and unde r standing of the drawing process ard its develop- 
ment, and a vivid illustration of retention and, indeed, development 
during an interval of no practice, I must stress that these children had 
no paper or pencil in the interval. For these case studies we had no 
additional information about the children; it was just not available. 

They came frurn a visually impoverished experience; there were vir- 
tually no man-made forms except huts. There were no trees --only 
rocks and scrubby grass and llamas and sheep. Although there was no 
previous experience with printed pictures or figure drawing, yet from 
the start in the older boy, we got recognizable form, and in the case 
of the younger boy, a study in the transition from scribble to represent- 
ation. 

I recommend to the art educator the cast 1 method, Even though you 
work with groups, you can collect case material on particular children. 
Even though you're very busy, you can usually find time to collect data 
for one case in which you happen to be particularly interested. An 
accumulation of such over a period of years can lead to process under- 
standing that group studies may conceal, as the following paper by 
Professor I3eitU 1 clearly reveals. 
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Fig. 3 




35 



42 



2 I £*E. iSSS 

CO 





36 

43 




Fig. 5 
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B, Some Values of Case Studies 



Kenneth R. Beit tel 



There are several points o r view one may take toward case studies. 
Sensitive experimentalists will frequently study single eases in some 
depth in order to develop an intuitive feel for phenomena they will even- 
tually have to represent by means of concrete operations and well- 
defined, quantitative variables. Others will immerse themselves in the 
center of the many vari bles encountered in ''reel life M or "in the field r 
in or do r to develop their hunches concerning which variables merit 
further, more controlled studies. These are uses which serve a pre- 
paratory function. The case study is a stepping-stone toward research 
or a means whereby sensitivity toward phenomena is increased, 

1 have no quarrel with such uses, but 1 Jo think the attitude toward the 
case study under such circumstances is one which relegates it to an 
inferior position for knowledge-getting. I will assume that there are 
other u:ret> of case studies which center value more within them, not on 
their a u \ i 1 i a r y fu n c t i on s , 

The forming of art can be seen as a process requiring an integration of 
socialized and idiosyncratic or personal, less ego-centered forces. 
Regularities in style, tradition, symbolism, and medium usage belong 
to the socialized side and submit gracefully to modes of inquiry com- 
mon to the behavioral sciences, I have been able, for example, to 
study characteristics of drawing strategies (with factors influencing 
them) of de fined sample groups. The noticeable qualitative differences 
between drawings of an individual, however, soon lead one away from 
the group focus and the variables and modes of analysis appropriate 
thereto. The socialized aspects do not cease. Rather the idiosyncratic 
ones press to the fore. More than that, it is the integration of these in 
the unique person which is the chief seduction of the case method for 
nu . It is my feeling that the aesthetic and creative aspects of art- 
forming do not submit to generic viewpoints or criteria, but that tin y 
are experiinti.il labilities dependent upon the flow oi articulation 
processes and upon the contexts in which these are embedded. 'I his is 



*’ s The reader is urged to consult Gordon V' , Allport’s, The Use of 
Pc r smal Documents 1 n Psy ch ologic a 1 Sc i o nce ( 3) for an e x t c nd c d dis- 
cussion of case methodology which is admirable in its scope and 
balance. 
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a philosophical assumption, i realize, su 1 will not argue it hero but 
merely state it, 

I'he re is a practic al reason as well as a philosophical one lor the study 
of the single case in art. The teacher assumes that his experience is 
a sound basis fur interpreting the student's artistic processes and 
products. He arrives at a necessary humility in discovering how little 
he can legitimately interpret of another's art processes, The effort of 
nonjudgmental interest and attention required to focus un the wholeness 
of a case engenders the empathy and insight which cancels out Ills 
categorical expectations and preconceptions. He perceives that student 
as a lawful entity whose images, intentions, skills and circumstances 
arc highly relevant to what he docs in art. This might be called an 
aesthetic or appreciative function of the case method; the grasp of 
wholeness, origin, and continuity, of existential being and transcend- 
ence circulating around the continuous making of art. Further, the 
teacher learns to what degree his assigned problems and curriculum 
may be impositions on a student, deflecting rather than helping him, 
Still further, the teacher may arrive at a role much like that of the 
therapist (although not oriented toward therapy) where he foul a himself 
mysteriously like a participant in a human dialogue which helps the 
other person direct his art dialogue in ways move satisfying lo him. 

To operate thus in the study of the single case and yet be a teacher may 
cause some confusion of roles. The observer whose chief interest is 
that of under standing the ease is likely to be in the belter position. On 
the other hand, the teacher who engages in such study may well modify 
his teaching methods to the place where there is little incongruence 
between the two rules. I have seen art teachers work effectively in 
this way, and there is a long-standing tradition in art education based 
on a belief in the organic "unfolding" of a person's expressive and 
artistic powers. It is not chance that Lowcnfeld and Schaeffer- 
Simmern, for example, have often included individual case histories in 
their writings, by so doing they thought to present subtleties beyond 
their generalizations through a more or less "ideal type" which their 
generalizations could then refer to. But I have found, in my in-depth 
work with individuals producing an extended series of drawings, no 
uninteresting types and no ideal types. Rather the full flavor ei 
uniqueness is expo licenced , along with the realization that all general- 
izations concerning a ease are but tentative expl mations and projections 
uf the observer. 

The case method in art, thus, leads one toward a phenomenological 
stance, toward philosophical questions about the experience of art, and 
away from group analyses and the kinds of manipulations more common 
to traditional behavioral science, it reveals to the teacher the way his 
students experience art, and it siusitiz.es the researcher to the full 
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range and complexity of the phenomena ho studies. C<unj histories, 
moreover, can bo structured so that they yield protocols which can be 
used in traditionai research methods. Allport, whose classic work on 
personal documents was earlier cited, also describes how data from a 
single case can be analyzed, placing emphasis on what he terms the 
M rno rphogenic" instead of the nomothetic. (2 pp 40 5-422) It is my 
hunch that such approaches as lie describes arc ideally suited to the 
study of art. 

With persons beyond the child level, first -per son singular statements 
about feelings, evaluations, intentions, images, technical problems, 
etc., are essential to an understanding of what the artist is doing. To 
elicit such statements requires, in my experience, a climate much 
like that described by Rogers (18) for psychotherapy, where the 
genuineness (congruence between statement and feeling) and empathy of 
the therapist, along with his unconditional positive regard for the 
client, are prime requisites. As in psychotherapy, to enter fully into 
this privileged arena is something of a trust and also an encounter not 
to be taken lightly. Any desire to manipulate the other seems out of 
place. 

In my own work, I hnvr interacted with the artist only between draw- 
ings. Usually the ' 1 inquiry" takes pLnee before a drawing session, 
using in-process photos of ihc previous session's drawing as a basis 
for stimulated recall and exploration* The photographic in-process 
material, the finished works, the notes J and my assistant make each 
session, and transcriptions of the inquiries add together to form a rich 
picture of a person guiding and transcending himself in art. In addition, 
without using the verb ‘to teach 1 ' in a transitive, pre-thought sense, I 
cannot escape the feeling that the method described is a good one for 
learning in art. Certainly the artisths self-identity and motivation arc: 
increased thereby. 1 have insisted that these cm so hi .lories constitute 
a Kind of abstraction and ’'knowledge 1 ' which is a valuable addition to 
art education. Assuredly they keep one humble and a believer in a 
kind of humanism rather rare in today's educational world. They sug- 
gest that the workaday ego is a small and often misleading represent- 
ation of the polemic lilies of -my giv n pmumn. 
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Identifying and Stating a Ri „-aM j-i. 1 ■ 1 il Problem 
Dale lb Harris 



Many, perhaps the most, reseurchable problems come from questions 
\vc raise during the normal course of our teaching or our work with 
teachers in supervision or in our discussions with colleagues and with 
students. It is perhaps the rare problem that one identifies by sitting 
down and thinking about his area of work until he goner, tes an ''aha! 11 
experience, locating the significant rosea reliable issues. Mure often, 
a problem may occur in tlm course of professional reading, in reading 
professional discussions or controversies, or studying reviews of 
research in particular areas, As one studies research, frequently 
questions arise concerning the unanswered questions of another 3 ** 
research. Other problems are identified from review of the literature, 
locating areas of work which have been infrequently investigated. In 
this latter case the rationed or ’'sitting down and thinking it through" 
approach may bu quite appropriate. 

Formulating a Problem SlnH incut 

Formally stated, the process of identifying and stating a rosea reliable 
problem goes through the following stages or stops: 

1. Stating the general question as it initially presented itself, 

2. Clarifying this question; defining terms and making it as 
specific as possible. 

3. If the question is broad, developing a number of specific 
questions that, if answered, would contribute toward answer- 
ing the central question. 

4. Distinguishing among t he issues those which are descriptive, 
those which are normative, and those which are experimental 
in character, 

5. Deciding wluil kind.', o! observations will provide the infor- 
mation necessary lor nrlsv. ering (nr beginning u> ansver) the 
quest ion. 



A Sample Question 




From among questions raised in research workshops, two may be 
selected as illustrative. One of these as originally stated was, ''What 
is the nature of the ‘floundering 1 some beginning painting students go 
through? Do certain psychologic, * conditions, such as anxiety, 
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confusion, hostility, fear, or rejection exhibit themselves to a certain 
degree in those students who identify themselves as flounde re rs ? " A 
second question was, "How can we ’free up 1 those r.hiLdren whose 
drawings and paintings are too constricted?" 

Restating thu Initial Question 

Taking the first question, one must break it down by asking additional 
questions cr restating and dividing it into separate issues and then 
deciding the :>rder in which one will seek the answers. For example, 
the first question raised above, calls up a number of further questions, 
such as, what arc the behavioral indicators that the investigator be- 
lieves will indicate floundering? Do students themselves know the 
experience of floundering? If so, are those who so identify themselves 
'he same students that the instructor would name as " t'l ounde rer s ? " 

Implicit Assumptions 

It is well at this point to recognize that the question implies several 
assumptions. One certainly is that the studen's arc mature, articulate, 
and well motivated individuals who are not simply "goofing off" be- 
cause they dislike painting instruction. Clearly, a. id particularly if 
the a rt students wen- school children, we would have to separate the 
well motivated from the poorly motivated children whose indifferent 
approach to their work would con u. c the is me. Secondly, such a 
question assumes that the investigator has had sufficient experience to 
identify the behavior of "floundc ring" and already know some of the 
overt signs of floundering. That is, one assumes that the investigator 
has a body of experience which can direct his initial observations; he 
will not observe all the behavior in the painting class - -merely those 
aspects which relate to "floundering," 

Formulating Specific bubstudies 

It is also necessary to recognize tl m the question very quickly breaks 
down info a number of distinct parts. One study would ascertain 
v. lit* the i slucenis themselves rev aware of the experience of flounder- 
ing, and whether these students arc the ueo tine teacher would 
identify from his nose i v.i tiem. Another would be a survey comparing 
two types of students, '1 Ik. btliaviors of f] ounde re rs and non- 
floun.de rers would be compared with respect to such psychological 
cha racte ri s tics a s anxiety, confusion, rejection, hostility, fear, and 
the like. At this point it would be well tor the investigator to talk with 
one of his psychologist friends, to identify acceptable behavioral 
indicators of the above nouns designating emotional states. This step 
is the necessary one of definition, and the selection of criteria. One 
must decide whether he will look for overt signs of these emotional 
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conditions in classroom behavior directly accessible to the teacher or 
will use psychological tests to get at dispositional traits. The psycho- 
logist may suggest certain personality tests which are sufficiently 
valid to investigate select eel dispositional traits or characteristics. 

Here one must be cautious as, despite a great amount of rest-arch, 
there is stiLL much confusion and debate in psychology as to the nature 
of dispositional traits. 

Returning to the broader question as initially slated, it is clear that 
the person who raised this question wants to know whether perhaps 
something other than emotional conditions contribute to the flounder- 
ing, Is some of this floundering in the nature of trial and error 
exploration rather than indicative of dispositional qualities or source 
traits? Clearly one aspect of the study should involve introspection 
and self report by students. Thus, inve stigat ioj . should focus on the 
process of learning, as well as on characteristics of the learner. 

In any event, the investigator needs to spell uut rather c .refully what 
he means by flounde ring - -the specific behavioral characteristics that 
he would identify as indicating this state in the learning of artistic 
skill, What about the following - -a sking the instructor many questions; 
making many false starts, and using many sheet -s of paper; wandering 
about the c assroom; looking at the work of other students; expressing 
dissatisfaction with the media; offering a variety of excuses for nest 
getting to work and staying at \vork--are all these signs of floundering? 
What other behaviors could signify floundering behavior? 

A Checklist of Observable behaviors 

First, then, the investigator must identify in behavioral terms the 
condition of ''floundering 1 ' as he defines it. Probably he will want to 
discuss these signs with colleagues to make certain that he has 
described a condition which other teachers also identify. Being able 
to state a commonly recognized situation gives some assurance that an 
individual is working with behaviors that have meaning and significance, 
and is not just expressing his own private concepts or peculiar inter- 
ests. 

Having satisfied himself that he lias defined a meaningful clutter of 
criterion behaviors, the investigator next will want to make a list of 
the specific actions that students exhibit when they ‘’flounder .' 1 He 
will also want to list a wide variety of illustrative verbal statements 
that students may make while "floundering," To provide those he will 
either draw on his store of accumulated observations or he will watch 
his class with an open eye and mind for a few days, making notes on 
the kinds of behaviors and statements which would lead him to say that 
students are floundering. Having Hsted such behaviors and statements 
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he would next ditto a preliminary checklist, arranging the items in 
convenient and logical groups. He would try out this checklist by 
observing for five or ten minutes each, several students whom he 
would identify as "typical f lounde re r s , 1 r making all the behaviors that 
he observes and adding any which lie had not included on the checklist. 
He would also wish to spend equivalent time wiih tumu typical non- 
ftoundercr to we Lint in fact r<d, »t iv. ■ 1 y few if -my of the behaviors on 
this checklist can be noted. 

Having obtained a workable checklist, he v. on Id then proceed to system- 
atic observation of all students in a given class. This task might 
require several periods of class time in which ho would observe e'ich 
student in turn for a short, specified time interval, checking off all the 
specific items of behavior which occurred in that interval of time for 
that student. He would go through .ill students in turn using equivalent 
amounts of time for observation for each student. On the basis of this 
simple checklist he could probably identify a slated number of students 
who show the most behaviors, and a similar number who showed the 
least behaviors in the t? ne interval, Thesa groups would constitute 
his contrasted gioup- of Hoiuider* r •; mid m >:i - fj omide r e r s. 

His re. -Nt might be to mu rvit w lius r r-tudi. uls to get their subjective 

e :<pe r i * ■ :ia s to :>ub.\t tnt hit < his ob^e rve < ion > , or he might pnKaM.-d to 
give p s v e h i d r )gi c. : i U.-sts s ‘leetid wills tin. advice ol colleague, as 
pr e v i ou s ly noted. 

It is well to recognize that in the above description we have involved 
several specific research tasks. We have defined terms, identified 
behaviors and verbal statements, conducted a simple survey which has 
certain normative characteristics to it. That is, the investigator has 
observed a group of students, each student for a similar period of time 
under similar conditions, ami checked off a list of behaviors noted 
during that interval, lie could make 'normative statements" about the 
typical number of floundering behaviors that arc likely to be exhibited 
in a short period of time in a < lass for beginning painting. He would 
have established a basis for further surveys of contrasted groups. He 
Would also hive it basis in*- ideut i t’vir.g a group Co* an experiment to 
lest vhnh t< ihniqu-'S might rcli.i c tm* mm-unt of 1 lounde ring behavior. 
Such an experiment would inquire two groups of students equivalent in 
3niti.il l lounde ring behavior, to be nwated, cue with special teaching 
techniques designed to redact floundering and the other, the control 
group, in which no special techniques arc used. 



Take the second question. "How can I free up children's paintings and 
drawings?" As staled, this is a methodological question, not u sub- 
stantive research issue. Certainly ii is not a problem yon can make 



A Second Kxample 
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observations on very readily. You wish to do something to achieve an 
objective, the objective being to make ciiildren's art work less stero- 
typed, more free and open. Again one starts with an assumption- -that 
freedom in art expression is a good thing. This is a value statement. 
Indeed, there is a very respectable bony of theory in art education that 
holds that freedom is a desirable tiling, and there’s much discussion 
in the profession'll literature concern nig how this objective can be 
a c h i e v e d . 



Onto running C rile ria 

For the descriptive portion of the research, the problem is to identify 
those aspects of drawings that are free vs. those that are stereotyped 
and constricted. So immediately we need criteria. Where does one 
get criteria? On? can, of course, draw on theory. What kind uf 
evidence would you look for iii children’s graphic art work that would 
indicate whether it’s constricted anil stereotyped or free and open? 
Someone may say -- 11 The use of space and the nature of line. M This 
statement is insufficient • Specifically, what usages of space and line 
suggest inhibition or frecdon? I suppose most of us would agree that 
small figures, placed sparingly on a sheet of piper, or perhaps 
restricted to one small area of the sheet would suggest const riction . 
Lanes made slowly uni carefully, or perhaps lightly, uncertainly, or 
tentatively would also suggest "criteria 11 ; already \vt are approaching 
verbal descriptions which should ui ibl< us to ti ll, when wo look at a 
drawing, whether or not these criteria apply. 

One might also consider that "movement" in the drawing has some- 
thing to do with openness or constriction. Indeed drawings have been 
evaluated quite successfully according to the general criterion: 
"movement depicted or implied." one might say, "It’s the number uf 
straight lines vs. the number of curved lines , 11 One might use such a 
criterion. Suppose, for example, one finds that the general rating ui 
movement in the drawing is actually highly correlated with the pro- 
portion of curved to straight lines. Then lie might conclude that these 
two criteria are really oidy one. But such a fact must be determined 
empirically; if you did find it, you'd need lo u»e only one of those 
criteria, rather th in both. 

One might also look a l themes- -conventional vs. original theme- in 
the content of art work. The notion is that conventional themes 
represent inhibition or const riction, and that unusual or "different" 
themes represent a form of freedom. In this we have another potential 
criterion, based on the idea expressed by the content, If it correlates 
only modestly with our other hypothesized criteria, we would have a 
basis for continuing to use it. 
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Having established, from theory, some possible criteria, \vc must 
ascertain that various judges can observe them similarly. Our judg- 
ments of criteria must satisfy the public nature of science; they must 
show agreement, or be reli a ble . Therefore we must test out our 
criteria by having two or more judges independently classify a sample 
of drawings or paintings by these criteria, to establish that their 
judgm nts are in fact similar. 

A Possible Experiment 

What we've done is establish some criteria by which we can judge work 
to be constricted or free. However, we still have the question- -how 
does one free up children's drawings? This now becomes a method 
question, suitable for an experiment. Many art teachers have rather 
definite ideas about how such may be accomplished. The investigator 
will try one of the methods suggested for "loosening up M children's 
work. He proposes to introduce a particular stimulus situation, to 
exercise a certain kind of instructional control, in order to produce in 
students 1 work a greater amount of the indices that lie has identified as 
denoting freedom. 

The experiment is science's most powerful technique, and it Leads to 
the most useful and significant knowledge. Experimentalists often dis- 
dain descriptive research, but unless one has gouu description, he can't 
experiment, because he doesn't know what he is working with. The 
experiment comes relatively late in scientific development. It may 
well be that in the field of art education there are some areas that arc 
ready for experiment, but there are surely many areas in which much 
more has to be known descriptively before we can identify what we want 
to do experimentally. Educators in art should not feel backward about 
simple descriptive work and hasten into expc rimenal work. There is 
plenty to be done in order to become familiar with the dimension of 
problems at a descriptive level. 

We have said earlier that observation is only one part of research 
design and concern. In observational studies we ask "What happens ?' 1 
and we can give descriptive accounts; we can make continued obser- 
vation.-? or limited observations; we can select critical instances; we 
can make process accounts, etc. Surveys answer the question "How 
are variables distributed in a population?"; and for surveys, as we 
indicated previously one uses interviews, questionnaires , schedules of 
items, or questions. The research problem of the survey is less that 
of accurate observation and description and more that of the sample- - 
sampling subjects in such a way that one can make general statements 
from the sample tc some theoretical larger peculation. 



Correlational Studies 



With instruments constructed for survey research- -into rviews, 
questionnaires, and schedules of various kinds (and remember that 
so-called personality tests are not tests of skills; they are really 
questionnaires or interviews) --one opens up a third area of work: 
Correlational studies. The research question now is "Mow are vari- 
ables associated with one another?" One may work with simple 
relationships, or he may work with multivariate relationships - -a 
number of variables intercor related. One may extend this procedure 
into factor analysis to examine, in a complex of intercor related 
variables, how many principal independent components or dimensions 
make up the complex of interrelated factors. Or one may work with 
prediction--by correlating variables (from which one hopes to predict) 
with particular outcome variables. Here, the problem of criterion 
measures is especially crucial, and \ve move immediately into the 
technicalities of measurement. 
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An Introduction To Measurement 



by William Rabinowitz 



To understand a person you must first be able to describe him and his 
behavior. This is equally true of teachers and children. To under- 
stand what takes place m the classroom it is necessary to describe it. 
This is also true of the products of behavior such as drawings and 
paintings. To understand such artistic products you have to be able to 
describe them. 



We have been discussing various procedures for observing and 
describing the person, his behavior, and the products of his behavior. 
It should be clear that a description ideally ought to be con plete and 
accurate. W e hav c s e on, h owe ve r , tha t n ny de script! 01 1 of a pe r s on i s 
necessarily limited to a tew of the many aspects of his behavior that 
are observable. Description is therefore t elective. 



In i jnrt, the selectivity of description involves a decision by the obser- 
ver concerning what to observe; in part, it involves a decision about 
how to observe it. Naturally, we want to observe those aspects of an 
individual's behavior that are especially relevant to car purposes. Our 
description of these relevant aspects may be relatively unstructured, 
impressionistic, and qualitative, or it may be relatively structured, 
systematic, and quantitative. For many reasons, descriptions term to 
be more useful as research data when they arc, or can bn, expressed 
in quantitative terms, To the extent that \vi succeed in making our 
descriptions quantitative and precise, we become involved in measure- 
ment. 

Most people, when they think of measurement, think only in terms of 
pape r -and -pencil tests. Although, such tests are an important facet 
of measurement in education, measurement implies far more than 
testing. In conducting descriptive research, a bash; requirement i.> 
the development of adequate procedures for observing and describing 
behavior. Such descriptions ol behavior may be considered measure- 
ments --sometimes crude and always impe rfect, to be sure, but 
measurements nevertheless. It seems appropriate therefore lo con- 
sider, some fundament il concepts related lo measurement 

Fundamental Concepts of Measurement 
An initial requirement in all measurement is defining the attribute or 
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property to be measured. If a property or attribute can be defined 
unambiguously, it can be rneasured--at least in principle. Notice that 
we do not directly measure a person or object. Instead, we measure 
certain properties or attributes of persons or objects. We measure 
the weight of a box, the height of a building, or the area of a field, Or 
we measure the intelligenc e of a child, the emotional climate of a 
classroom, or the c rcativity in a work of art. 

In many instances, particularly when we are dealing with simple 
physical attributes, the meaning of the terms wc use appears self- 
evident, We don T t typically see any need to define what we mean by 
weight, he ight , or a rea . But when we want to study attributes that 
are of educational or psychological interest, we encounter d finitional 
problems almost immediately. Psychologists - even after years of 
study - cannot agree on definitions for intellig e nce , emotional climate, 
or c reativity . 

You may have noticed that when you try to isolate md describe some 
interesting characteristics of drawings or paintings, the task presents 
certain problems. Many of these problems are directly related to the 
terms you use to describe the drawings - terms such as balance, 
tension, texture , rhythm , etc. The referents for these terms are not 
obvious. The attributes of the drawings to which these terms refer 
are not immediately apparent. Often you may find yourself challenged 
by your colleagues to define your terms. And often your definitions 
are not accepted by other members in your group. To the extent that 
you cannot specify clearly what you mean by the attribute -te rms you 
use, it is a fundamental stumbling block in your efforts to do descrip- 
tive research. 

I said earlier that descriptions are most likely to provide scientifically 
useful data when they are based or* systematic obsc rvational procedures 
and c re expressed in reasonably precise and quantitative form. Insofar 
as r o ate aide to describe poisons, objects, or events in this way, we 
are engaged in measurement. 

Scientists have somewhat different conceptions of what measurement 
is, but among all of these varying conceptions there is a common 
idea. Measurement is the assignment of numbers to persons, objects, 
or events in accordance with certain rules. Whenever you have a rule- 
or a set of rules - that you can employ in assigning numbers f o the 
phenomena you are observing, you are measuring. 

The rules scientists use to n ..sure vary, and this variation leads to 
the important concept of scales of measurement. Consider for the 
moment some of the familiar properties of numbers. Numbers can 
express equivalence or difference. Thus, 1 is the same as 1, but 
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different from 2 , Numbers can express order of magnitude . Thus, 1 
is less than (or comes before) 2 , and 2 in turn is less than (or comes 
before) 3, Numbers can express ^quality of differences . Thus, 80 is 
as much greater than 70 as 70 is greater than 60, Finally, numbers 
can express equality of ratios . Thus, 80 is twice as large as 40 and 
40 is twice as large as 2U. 

Each of these conventional properities of our number system has a 
parallel in the scales of measurement we are going to discuss. And 
the exact nature of this parallel depends upon the rules we employ 
when we assign a number to a person, object, or event v/e are trying 
to measure. 



Nominal Scales 

Consider the simplest situation. As an example, we assign a group of 
persons to two categories, n Male" and "Female" and we call the males 
"0" and the females "1 M , The rule for assigning numbers here is 
clearly a very simple one. Each person can be unambiguously assigned 
to either the M Q M group or the ”1" group depending upon a particular 
property he (or she) possesses - namely his (or her) sex. All of the 
0's are equivalent to one another with regard to this particular 
attribute, and all of tlie l*s are also equivalent to one another with 
regard to this particular attribute. But none of the 0‘s are equivalent 
to any of the 1 1 s with regard to this particular attribute, 

I could give many more examples of this type of measurement. We 
might, for instance, classify paintings into categories such as M oils, M 
"watercolors, M and "pastels” ind use the numbers n l M , ”2”. and ”3”-- 
or any other three different numbers --to designate the categories. Or 
we might classify schools into the categories, ’’Public”, ’’Privatc- 
Parochial”, and ” Private -Nonpar ochial, 11 Once again we could use 
three different numbers to designate the categories. In each of these 
examples it is easy to recognize the attribute we are trying to 
measure, and the rule we would use in assigning numbers to the 
objects - -paintings or schools - -being measured, 

It may have occurred to you that this process doesn’t really resemble 
measurement as we usually think of it. If this is your reaction, you 
are in distinguished company, Many scientists are also unprepared to 
call this use of n* nbers measurement. However, it is frequently 
referred to t s a nominal scale , and I think it is useful to discuss it, 
although whether it should be considered measurement is debatable. 

The numbers in a nominal scale simply indicate that two objects are 
equivalent or not equivalent in some respect- -and nothing more. The 
distinguishing feature of a nominal scale, as you have probably 
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noticed, is the use of numbers as substitutes for verbal labels. Num- 
bers used this way must be interpreted very cautiously. For example, 
the numbers that appear on the backs of football players constitute a 
nominal scale, We can be sure that "IS" is a particular player with a 
particular name. In the same way, 11 tO’ 1 is another, different player 
with a different name. The two men are different and they have 
different numbers as they should on a nominal scale, But notice that 
we can say nothing about the t.vvo men- -other than that they are dif- 
ferent- -by examining the order of the two numbers of the difference 
between them. 



Often we can recognize not only that objects are equivalent or not 
equivalent to one another; we can also recognize that they can be 
ordered or ranked in seme way. Thus we may observe five different 
art lessons and feel that we can order them on a characteristic such 
as, "Structure," We would assign a rank of M 1" to the lesson that was 
most structured, a rank of "2" to the lesson that was next highest in 
structure, and so forth. This procedure for assigning numbers in- 
volves a different rule from the one that defined a nominal scale. Here 
we can discriminate order, not only difference. In effect, we are 
saying the event to which I assign a " 1 1 is not only different from- -it 
i.? also higher than- -the event to which I assign a "2 n , A scale produced 
in this way is called an r - dinal scale. 

Ordinal scales are very common in every day life. When a movie 
reviewer describes a picture as a "3-*: tar' 1 picture and another as a 
"4-star ,r picture, he is using an ordinal scale. His us *3 of the scale 
implies that any movie called a "4-star n picture is better than any 
picture called M 3-star, M Thus, the movies are being ranked or ordered. 

Ordinal scales are also very common iu educational and psychological 
work. The conventional letter grades, "A", rf B", "C", "D", and "F" 
with which we assess academic performance constitute an ordinal 
scale. The scale implies that any student who receives an "A" in a 
particular course is a better student in that course than any student 
who receives a ;, B" and so forth. Though we use letters in assigning 
grades, we could as easily use numbers such as "4", "3", ' 2", "l", 
and 1r 0" -- and this, of course, is exactly what we do when %ve com- 
pute a grade-point average. 

Probably the most common exaniji^s of ordinal scales in educational 
and psychological work are the ubiquitous rating scales with which we 
try to measure everything from teacher effectiveness to pupil person- 
ality. Usually these scales have five steps - -sometimes or two 
more or less --with a 1 ! of the step'; described so that they fall in J .o a 
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clear order. Thus, I could develop a scale to rat? teacher -pupil 
rapport in the classroom which might look something like this: 



Teache r - T:> apil Rapport 

1, Teacher and pupils work together in complete 
harmony; there is no evidence of any tension. 

2, Teacher and pulls work together in good 
harrpony; there is little evidence of any tension. 

3, Teacher and pupiLs work together in fair har- 
mony; there is some evidence of tension. 

4, Teacher and pupils do not work together in 
harmony; there is much evidence of tension/ 

5, Teacher and pupils are in obvious disharmony; 
there is evidence of almost constant tension. 

This scale would clearly be an ordinal sc?Fie, You should recognize 
that none of the distinctions among the five steps in this rating scale 
indicates how mu c h bett er the rapport is in one class than another. 
You can say that in a class rated "2 U the rapport is better (or should 
be if the rating is accurate) than in a class rated n 3". But you can- 
not say that it is as muc h better as a class rated M 3 ,r is better than a 
class rated 11 4' T . 



For some problems you must be able to show that equal differences in 
the magnitude of the events being measured are associated with equal 
intervals between the numbers assigned to these events. Thus, you 
must show, for example, that if you measure three events and they are 
assigned the numbers u \5", 11 1 0 M , and u 5" then the difference between 
15" and '4 0" is the same as the differ ence between "10" and "5 ,r . In 
effect, you must be able to show that a 5 -point difference represents 
the same thing everywhere on the scale, 

When this condition prevails we are using num K rs to indicate more 
than order. If we have demonstrated equal 'ty of intervals, we call t He 
measurement procedure and intcwal sci-lg. A familiar everyday ex- 
ample of such n scale* is the Fahrenheit scilc for measuring temper- 
ature. The difference between 20 F' an 3 30 F is the same difference 
in temperature as that between 40 F and 50 F or 60 F and 70 F. 

There are not too many characteristics of educational or psychological 
interest that wc typically measure r sing interval scales. Consider 
intelligence as an example. It is frequently assumed that the IQ scale 
is an interval scale, but is thett so? Is the difference between an IQ 
of 90 and 100 the same difference in lrtelligence as that between 60 
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and 70 or 140 and 150? Does a difference of 10 points have the same 
meaning, as a difference in intelligence , at all points on the scale? 

Wc often assume that it does, but that is only an assumption. Even 
when intelligence test data are converted to normalized standard 
scores, it is an assumption --net a demonstrated fact --that equal dif- 
ferences in standard score iO's correspond to equal differences in 
intelligence. 

Recall the discussion by Dr. Harris of the procedure we used to study 
the drawings of prospective teachers who were asked to ’'draw a 
picture of a teacher with a ctaso." ( 1 6 ) We developed scales to 
measure in the drawings such characteristics as ’’Relative Teacher 
Emphasis," "Psychological Distance," 11 Teacher Initiative, " "Tradi- 
tionalism in Classroom Arrangement. " and "Artistic Quality." Each 
of these scales is defined by a set of five sample drawings numbered 
from "0" to ’'4 rl . In selecting the sample drawings we went to a great 
deal of trouble to select drawings :hut were equally distant from one 
another on the attributes being measured. Wc did this because we 
were interested in measuring 'he five attributes on interval scales. 

We could, of course, have developed ordinal scale: with very little 
difficulty. We made an additional effort to develop interval scales 
because such scales, when they can be developed, permit more percise 
measurement and the use of more powerful statistical methods in dma 
analysis . 



There is a final type of measurement scale called the ratio scale. 
Certain events occur which match the number system not only in terms 
of increasing order and equivalence of intervals, but also in terms of 
the existence of a meaningful zero point. When a meaningful zero 
point exists in some att ribute - -we often say a true zero point--then it 
is possible to discuss ratios meaningfully , It is possible to say that A 
is twice, or one -half of, B on the attribute being measured. 

The most familiar examples of ratio scales are the scales we use to 
measure such physical properties as length and weight. A string 5 
inches long is exactly one-half as long as one that is 10 inches long. 

A box that weighs 20 pounds is twice as heavy as one that weights 10 
pounds. We can make these statements meaningfully because in each 
case a unique and meaningful zero point exists. If we return to our 
earlier example, of the Fahrenheit temperature scale, you will note 
the importance of the requirement of an unique or true zero point on 
the scale. The Fahrenheit scale ha s a zero point; it happens to coin- 
cide with the temperature at which water freezes. It is not a true or 
unique zero, since there are clearly temperatures far below this 
point - -represented as negative varies. And for this reason it is not 
a ratio scale , 
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Ratio scales are rarely achieved in educational or psychological 
measurement. Consider intelligence once again as an example. Is 
there any sense in which we can say th? c a parson with an IQ of 150 is 
twice as intelligent as one with an IQ of 75, We can, of course, say 
that two boxes, each of which weighs 7 5 pounds, weigh together as 
much as one box that weighs 150 pounds. But clearly two individuals 
witli IQ's of 7 5 each cannot in any meaningful way be shown to be 
equivalent to one individual with an T Q of 150. The IQ scale may be an 
interval scale, but it surely it not a ratio scale. There is no unique 
and meaningful zero point in intelligence. 

This, of course, is almost certain to apply to our efforts to measure 
characteristics of major concern to art educators. A meaningful 
zero point in the measurement of creativity, for example, is, I think, 
impossible to conceive. To the extent that 1 am right, it will not be 
possible to develop a ratio scale to measure creativity. And a state- 
ment such as 11 A is twice as creative as B 11 will be meaningless, 

Let me try now lo summarize very briefly what I have been saying 
nb Tut scales of measurement. We defined measurement as the assign- 
ment of numbers of persons, objects, or events in accordance with 
certain rules. Depending upon the rules employed, we may measure 
using a nominal, ordinal, internal, or ratio sca:e. In most of the 
situations in which you as art educators are likely to be measuring, 
you will probably be using nominal or ordinal scales, In some cases 
you may be working with interval scales, but that is not likely to be 
very frequent. Except when you measure a painting’s size or the 
weight of a piece of sculpture, you will not have many occasions to 
employ ratio scales in your descriptive reseaic’n. 



Let us assume now that you lmve a measurement procedure. Perhaps 
you are measuring a.spccts of behavior in the classroom or some impor- 
tant characteristics of children's art products. One of the problems 
you encounter almost immediately is the measurements are not 
completely accurate or dependable; they are subject to wliat is called 
error. We have to spend some time discussing error, because the 
presence of error in measurement means that scores are not, and 
cannot be, completely reliable. 

We begin then with the recognition that no measurement is perfect; 
all measurement contains error. And this is true of all measure- 
ments -- in the physical sciences as well as the behavioral sciences. 
Fortunately for physical scientists, the magnitudes that they are 
usually dealing with are very large relative lo the errors of measure- 
ment, Since the error sizes are typically relatively small and the 
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magnitudes they are trying to discriminate are relatively large, the 
physical scientist can usually measure with a great deal of confidence. 

In psychology and education, the situation is very different. We are 
often trying to measure small differences in some attribute, but the 
measurement errors are relatively large. As a consequence when we 
detect a difference in our measurements we often cannot be sure if it 
reflects a true difference between the objects being measured or is due 
to error. 

In developing a measurement procedure, one of the first and most im- 
portant tasks you face is reducing the size of the errors of measure- 
ment as much as possible. To the extent that errors are very 
prominent in your measurements, they will not -- and cannot -- relate 
to anything. Research based on measurement procedures that are 
unreliable -- that is, subject to large errors - - is doomed to fail. So 
you must try to keep error of measurement as small as possible. To 
do this you must understand what error is and how it gets into measure- 
ments , 



As an aid to this discussion, 1 am going to make a distinction between 
systematic and random errors. Let me discuss systematic errors 
first. Systematic errors occur whenever you arc; measuring, and the 
numbers you assign to certain persons, objects, ox' .vents are system- 
atically either too high or too low. This could happen for many reasons 
and in many different ways. Suppose I am trying to measure the intel- 
ligence of everyone in this room. I do not have an intelligence test; all 
I have as raw data are my observations of your behavior. So probably 
I would have to rate your intelligence on the basis of these observations. 
Le^s say that I had a rating scale, an ordinal scale with five steps. 
Suppose further that the nature of m> measuring process is such that I 
tend to think of men as more intelligent than women, For the men, I 
tend tc resolve all of my doubts in favor of assigning higher ratings; 
for t lie women, I tend to resolve all of my doubts in favor of lower 
ratings. That :.s a systematic tendency on my part to rate inen too 
high and womcr. too low. On the perfectly reasonable assumption that 
intelligence is 'distributed equally between the sexes, any tendency on 
my part to judge men higher than women is a systematic error. The 
effect of this error on my me asm ements is h’ghly predictable once you 
know the nature of my uias. 

Suppose, as a furtlier example, you wanted to measure the extent tn 
which a group of teachers were able to establish rapport with pupils. 

You might ask the principals of these teachers to rate them on this 
characteristic. It is highly likely that the principals’ ratings would 
not be pure measures of rapport. The principals, being human, would 
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very likely give higher ratings to teachers whom tiiey found highly 
cooperative and lower ratings to teachers whom they found highly un- 
cooperative. This, of course, is a systematic error. The teachers 1 
scores on rapport are being systematically elevated and depressed by 
a factor that is unrelated to what you are trying to measure. 

In developing a measurement procedure it is important to anticipate the 
systematic errors that are likely to be operative so that their influence 
can be reduced. For example, suppose you were doing a study involv- 
ing observations of teachers in the classroom. If one of your observers 
tended to rate the pretty teachers higher than the not-so-pretty teacher; 
he would be introducing a systematic error into your measurements, or. 
the as. umptio.i that you were not trying to measure prettiness. Once 
you discovered this, you would have to do something about it. You 
would eithc 1 ' re -train this observer or drop him from your study. 

To pursue this example further, suppose the bias of this observer was 
very strong, Tret's say Mia t every time lie observes a pretty teachei he 
rates her "5" and every time he observes a homely teacher lie rates 
her 11 1", If this were the case, you could never discover anything about 
the classroom behavior of these teachers, because their scores were 
totally determined ty the observer’s bias. The systematic error is so 
large in this case that it completely overwhelms the property you are 
trying to measure. This, of course, rarely takes place, and when it 
does it is very easy to detect. What is more common is for much 
smaller error tendencies to operate. Though smaller, such error 
tendencies can nevertheless be large enough to reduce markedly the 
accuracy of your measurements. 

Another example of systematic error is something that occurs in a 
quite different context, and that is when people report -- tlirough 
questionnaires or interviews -- Jieir attitudes or feelings. In such 
situations, it is common for individuals to distort their responses in 
order to create a favorable impression. For example, suppose you 
were giving a personality inventory to a group of individuals, all of 
whom were applying for a job as a salesman for a life insurance 
company, and you had questions on the inventory like: "Do you enjoy 

talking to people? 11 and "Do you feel comfortable when you're trying 
to influence somebody to make a decision ? u Now consider an 
applicant for a position as life insurance salesman. How will he re- 
spond to questions like these? He will almost certainly answer M yes, u 
whether or not he feels that way. His scores are therefore going to 
be systematically distorted in the direction of appearing more like a 
life insurance salesman than would probably be the case if he were not 
taking the test for that purpose. 

The same thing can happen in a research study with teachers. If you 
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give teachers personality or attitude inventories to fill out, they are 
almost always going to try to put their best foot forward. We worked 
some years ago with a test called the Minnesot a Teacher Attitud e 
Inventory. It’s a test that has statements like, "Most, children are 
obedient, 11 The response options are "Si rongly Agree, M "Agree," 
"Unce rtaim " "Disagree, 11 and "Strongly Disagree." There arc ISO 
statements with five possible responses to each statement. In one 
study we gave this test to prospective teachers and then collected their 
test papers. Then we gave the test once again, but the second time we 
said, "Now take it and try to get the highest possible score, 1 ' When we 
compared the scores on the second testing with the first, we didn't find 
that there was much difference. As a matter of fact, there was almost 
no difference whatsoever. Tins makes you feel that the first time the 
prospective teachers took the test, they were trying to get the highest 
possible score. In effect, they were, faking a good performance. — 
distorting theii* "true" attitudes in order to earn as high a score as 
they could. This, of course, is a type of systematic error. 

Suppose yon want to observe and measure certain aspects of the typical 
classroom behavior of teachers. You might make it a practice to 
advise teachers well in advance of year visit of your desire to observe 
them ai: a particular time. This seems like an appropriately courteous 
action on your part, but it is likely to mean that the behavior you 
observe will, to some unknown extent, not be typical. It will almost 
certainly be systematically modified by the teacher in order to create 
a good impression. The teacher exhibits her best behavior which may 
or may not be typical of her. 

I noted earlier that rating scales are widely employed measurement 
devices in educational and psychological work. Though very popular 
ratings do not typically' yield very satisfactory measurements. Parr of 
the difficulty is that they are subject to at least two types os systematic 
errors. 

One of thes'* is generosity e rror, that is, the tendency of raters to 
give ratings thac are too favorable. Piincipals tend to rate their 
teachers 'r "Excellent 1 of "Very 3ood." Only the most trouble - 
some or inadequate receive low ratings. Such ratings are of little 
value siive they do not discriminate effective ly among the individuals 
beir.g rated. In general, genero^ty error is likely to be very promin- 
ent whenever supervisors rate suoordinates. And this suggests, in 
part, why ratings are too generous. The supervisor is likely to feel a 
greater loyalty to those with whom lie works -- no matter how inade- 
quate they may be -- than ho does to those who are requiring him to 
make the ratings. 

The other type of systematic error n ratings is called the halo effect. 
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To understand this type of error you must recognize that usually when 
ratings are employed, the rater is expected to rate individuals on 
several separate traits* The person who wants the ratings hopes to 
get several, relatively independent pieces of information about each 
ratee, And that is the reason for the use of several rating scales. In 
practice, however^ raters tend to be strongly influenced in assigning 
their ratings by the general, overall impression they have of each man. 
Thus, the rater does not usually make the careful discriminations 
implied by the necessity to rate a number of specific traits. Instead, 
the ratings of these specific traits tend to reflect the rater ! s general 
opinion of the individuals merit or worth. The halo effect means that 
ratings of specific traits are underdifferentiated or, to put it an- 
other way, too highly correlated with one another. In any event, the 
effect of this error is systematically to obscure the pattern of traits 
within the individuals being rated. 

Random Errors 

I could continue discussing sources of systematic error, but it seems 
aporopriate to consider briefly the other type of orr:r -- random error. 
You can recognize the difference between what I am calling systematic 
and random error by noting that systematic error produces a predictable 
effect whereas random error does not. The biases that are lumped to- 
gether and referred to as systematic error have a predictable tendency 
to elevate scores in some cases and depress them in other cases. 
Generosity error, for example, tends to produce scores that arc too 
high* 

Random error is inherently unpredictable. It occurs because of a 
myriad number of influences most of which are so poorly understood 
that we often say that random error is due to chance factors. We in- 
voke chance to "explain 1 ' random errors of measurement because in a 
fundamental sense it really cannot be explained. 

It is important co recognize, however, that random errors will have 
unpredictable effects, sometimes elevating, sometimes depressing 
scores. Thus an obtained score, derived from some measurement 
procedure, is distorted in some unknown way by random errors of 
measurement which cause it -- in an inherently unpredictable manner-- 
to be either too high or too low. 

We can illustrate some of these rather abstract concepts by refere nce 
to a hypothetical example. Suppose I am interested in measuring the 
effectiveness with which a group of art teachers conduct their art 
classes. As part of this effort, I visit Miss Suzy Jones on April 20 at 
9 A.M. for 30 minutes. At the end of my 30-minute visit, I give 
Miss Jones a rating and this rating icpresents her score as far as my 
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investigation is concerned. You can assume that the other art teachers 
are measured in the same way. 

It is probably very clear to you that this method for measuring a 
teacher's effectiveness leaves a great deal to be desired. Our 
measurement of Miss Jones, and the other teachers, is derived from 
a single observation. We know nothing about hov/ typically Miss Jones 
behaved on that particular occasion. Specifically, how can we be sure 
that Miss Jones was not atypicallv exciting or dull on that day? Since 
the observation was taken at 9 A.M. ,vc may assume that; Miss Jones 
is not a better, or worse, teacher in the. morning than in the afternoon, 
but do we know this? And since the observation was terminated at the 
end of 30 minutes, we are probably assuming that this is an adequate 
sample of Miss Jones' behavior. But Miss Jones may be one of those 
teachers who gets better, or worst;, as a lesson continues. Finally, 
how shall we interpret liie use of a single observer to visit Miss Jones 
and the other art teachers? Is the definition, of ’’effectiveness 11 in art 
teaching so unambiguous that competent observers will always agree on 
the score that each teacher should receive? 

The questions 1 have been posing suggest the sources of random errors 
of measurement. .Behavior is unstable; it changes fiom one situation 
or occasion to another. The measurement standards of different 
observer..; are not uniform; they vary from one observer to another. 
This means that a score based on a single observation by one observer 
is not very trustworthy. You cannot depend upon it to reflect the true 
status of the individual observed. 

This concept of "true” status requires some examination, Fet's go 
back to Miss Jones and the other art teachers. We were trying to 
measure their effectiveness, but what does that mean? Without try- 
ing to define "effectiveness" it should be clear that we are trying to 
measure a gene ral property of each teacher 1 * behavior* How any 
specific teacher functioned on any particular occasion - as observed by 
a particular observer - is oi no great consequence, since we are inter- 
ested in general behavioral properties, not specific events. Each 
teacher’s "true" effectiveness could be thought of as an average of her 
behavior across all possible occasions as obse wed by all possible, 
competent observers. Clearly, to measure true offectivene ss , con- 
ceived of in this way, is an impossibility. But how c.in we approximate 
it? It should be obvious that we need many observations by several 
observe rs , 

J hi s then is the principal way in which we reduce random errors of 
measurement --by taking a larger number of observations. This is a 
highly gene ral i /.able principle; it applies to paper -and -pencil tests as 
well as observational measures. /HI oilier things bcii.g equal, a 
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longer test is rriore reliable than a shorter test. Adding lost items 
reduces the relative contribution of random errors of measurer enl to 
individuals’ - core. , In the same way, a measure of an art teacher's 
ei ective lies? b co ncs more reliable as t is based on > h;iu,er number 
of observations 

This may be an appropriate point to conclude our discuss]. ... of measure- 
ment. I hope that it is clear that my comments were quiti 'wneral, and 
deliberately so. Measurement can be a very technical aiva, and to go 
much beyond my very general remarks will almost >uv« \y involve tech- 
nical concepts of increasing complexity. For example, we c nnot 
explore such concepts as validity or reliability without some dutisticul 
understandings. The going gets rougher as you continue. I hope, how- 
ever, that I have encouraged you to see measurement a eoi : .p rehensiw 
and interesting, and that you will continue beyond this pouit, 
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The; preceding pages have presented the major content that was the 
focus for the three days of each of the Research Training Institutes on 
Descriptive Research. The purpose of the publication is similar to 
that of the Institutes, that is, to inform and to encourage participants 
(readers) to get involved with research methods in their own situation. 

With all good intentions, those who have little or no previous back- 
ground in research methods, may still face, with some puzzlement, 
the question, ''Where do I start?' 1 At the risk of oversimplif ication, 
the following may be a way to begin. This is learning exercise, 
similar to one that was used on the first day of the Institutes. 

1, Collect a small number of drawings from your class (or someone 
else. ’s), at least 10 but not more thin 2 0, to start with. Work 
done on the same size paper will be more convenient. 

2, Lay them uo the floor or pin them on the wall, in any arrange- 
ment where you can look over all the work with case. 

3, You are now ready to make your first observations. With pencil 

and paper in hand, look uver all the drawings carefully and begin 
to list the attributes (charactei istics) that are present in some or 
all of the drawings. For exai :ple you may note such attributes 
as: subject matter differen s >s; some used shading, others did 

not; all had shapes but some had more than others, and so forth. 
A note of caution is important as you proceed. We are concerned 
at this point only with what \vc can. observe. You may be tempted 
io include inferences, such as 11 Some students didn't finish their 
work.' 1 This, of course, is an inference you have made from 
something you have observed but is nut the direct observation it- 
self. As you continue, it is likely that you will find that the task 
become s more involved than you had anticipated, i \ s you move 
fro n simple to more complex phenomena. For example, in your 
first observations you may have noted that all the works had lines 
in them and ihal some had more lines than others. A.-» you pro - 
cced you may decide th t having few or many lines seems less 
important than the qualities of lines, and you will begin to note 
line characteristics such as straight or c uved, light or dark, 
thick or thin, 

4, After you have listed 3 or 6 attributes, it will be wise to test 
these by sorting the drawings according to each of the attributes. 
You may separate all of the d innings that have shading from 
those that do not, At this p 4 t you may feel disturbed that there 



is so much variety in the pile of drawings with shading; some 
have very little compared to others. In trying to separate them 
according to the amount of shading -- you may find yourself 
thinking in terms of: n^ne; a little bit; a lot; all sha.ded. In do- 

ing so you have developed an ordinal scale that can be used for 
measuring the amount of ’’shading 1 ’ that is observable in the 
drawings » 

5. As the number of attributes increases, you are likely to en- 
counter a certain amount of overlapping of terms. For instance, 
you may find that a ’’light line” may refer to one that is "thin”, 
one that is ’’light in value”, or a combination of these. In order 
to be more specific you will need to define your terms in order 

to clarify their meaning. If you decide that a "light ]ine” a line 
that is light in value, your descriptions can become more precise 
since you can now refer to "light thin lines" or "light thick lines”, 

6, Eventually you will realize that the task seems unending and 
decide to stop. The exeredse is in theory an endless one increas- 
ing in precision as you discover attributes in the line detail of the 
drawings and define your terms. Having read this far, you may 
decide not to try this exercise at all, since you have jusl read 
about it. However, do not jump to this conclusion too quickly; 
reading about it is no substitute for the experience of doing it. 

This opportunity to test your own perceptions in discovering at- 
tributes, practice in clarifying meanings, and developing rating 
scales will pay dividends when you begin working on specific 
problems. The section by Dr. Harris on "identifying a Research- 
able Problem" will help you take the next step and you are on your 
way toward getting involved with research methods. 

Descriptive research is that category of research that attempts to de- 
scribe "the way it is" with more precision and accuiacy than one can 
obtain through casual observation. It is easy to say that each 'hi Id 1 s 
drawing is a unique expression, but by’ the recording and analyzing of 
care -ii systematic observations of children's drawings we can begin 
to s^rt out what similarities and differences exist. Whan we have done 
this, we can begin to verify and give more specific rm» nlng to the 
assumption we have expressed. However, in com pa -oon to pa A 
history, "the way it is" today may be only" a temporary condition be - 
cause of the rapid changes that are taking place throughout ail facets of 
our society". 

The need for ongoing descriptive research has never been more critical 
than it is at this time. Anyone who has been involved with public 
education for the past decade s well aware that the youth of today are 
quite different from those in school during the 1950 T s t But when we 
begin to discuss "how" they" are different, we rely* on imperfect mem- 
ories and describe in vague gene ralit . e s . Changes in art programs 
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and the development and inclusion of new art materials and new art 
forms is on record. Knowledge of changes in students and their effects 
on behavior in art remain speculative since V3 have liltle document- 
ation in this respect to make comparisons. 

No research of any kind will provide the answers to the "ought" ques- 
tions that we face every day. What curriculum changes "ought" we to 
be making to keep up with times? What knowledge "ought" students to 
be learning in order to cope with the problems of the future? What 
criteria "ought" we to use to evaluate s tude its? The answers do not 
lie in the findings of research studies, but a backlog of descriptive 
materials about students and student behavior in art can be analyzed 
for changes and developmental trends that can help provide a basis for 
making more intelligent decisions concerning these kinds of questions. 
Without more reliable information we arc left to rely on the fads and 
fashions of educational innovation. 

Art educators trained in research methods are relatively few, and 
their combined number can not provide the descriptive data that is 
needed. We can only hope that more interest and active involvement in 
gathering descriptive material will come from teachers and supo - 
visors working in the schools and colleges. 



82 

7 5 



BIBLIOGRAPHY 



1. A Seminar in Art Education for Researc h an a Curriculum Pavel - 

opm ent; Cooperative Research Project No. V - 0 0 2 ; 

Edward L, Mattil, Project Director, Pennsylvania State 
University, University Park, Pennsylvania, 1%6, 

2. Allport, Gordon W. , n T ho Unicr t md the General in Psychologi- 

cal Science", Journal of Pc rso nali ty, 1962. 

3. Allport, Gordon W e , Th e Use of Personal Documents in Psycho- 

logical ^cience Now York; Social Science Research 
Council, bulletin 49, 1942, 

4. Art Ed a cation; edited by W , P.eid Has tic, 6 1th Yearbook of tho 

National Society for the Study of Education <Parl II), 1963. 

5. Darker, R. G. , and Wright, II. l\ Midwest and its Children, 

Evanston, Illinois, Row Peterson n Co,, 193 4. 

6. Darker, R, G. , Ecolo gic al Psychology, Stanford, California, 

Stanford University Press, 196S. 

7. Eisner, Elliot W, , "The Development of Information and Attitude 

Toward Art a l the Secondary and College Levels", Studies 
in A rt Education volume 8, Number 1, Autumn, 1966. 

8. Eng, Helga, Th e Psychology uj Child ren's Drawings, London: 

Rout-ledge & Kogan Paul, 19 31. 

9. Eng, Helga, The Psycho lo gy of Chi ld an d Youth Drawing, New 

York; Hamanities Press, 1 957. 

10. E xc-mpla ry P rog rams m An Ed i.;atio:i; edited by Stanley t ti eje , 

National Art Ed ic a lion Association, I960. 

11. Goodenough, Florence* I, McasuroincuU of lnjen_ig< rice by Drawing, 

New YorK, Ha r court Brace, and World, 1926, 

12. Harris, Dale B. , Chi ki‘*cuijs Drawing s_as M ee sure s of Intellectual 

Matu r ity. New York: Harcourt Brace, 1963, 



o 

ERIC 



83 

76 



13, Hildreth, Gertrude, The Child Mind in Evolution: A Stud y of 

D e v e 1 opn lental SccimMii'e in D raw ing , New York: King's 

Press, 1941. 

14. LaPage, Geoffrey, Art and the Scientist, Bristol: John Wright 

and Sons, Ltd. , i 4 >G I . 

15, McGurdy, H. G,, "Group and Individual Variability on the 

Good enough Dravv-A-Man Tost 11 , Journal of Education 
Psyc hol ogy, 1947, 38, 428-36. 

16, Rabinow it z , W. and Travers, R. M., "A Drawing Technique for 

Stud ing Certain Outcomes of Teacher Education" The 
ouvual of Educational Psych oiogy , Vol. 46, No. 5, May, 
1955. 

17. Report ol the Coiiimission on Art Education: edited by J . Hausman, 

National Art Education Association, 1965. 

18. Rogers, Carl K. , ''Learning to bo", in Farber, Seymour, and 

Wilson, editors, Con Hut and Creativity, New York, 

M cG raw Hill, t C >G~37 

\ r K Sax, Gilbert.. Empirical b_oundalions of Educational Research, 

Engclwood Cliffs, New Jersey, Prentice -Hall , Inc,, 1968. 

20. Seminar on Elementary and Secondary School Education in the 

Vis.ua Arts: Cooperative Research Project No. V-003; 

Howard Coiuuil, Project Director, New York University, 

New York, N. Y. 1965. 




84 



77 



APPENDIX 



A. Selected References 79 

B. Preconference Program 81 

C. Institute Evaluation, William Rabinowitz 83 

D. Pre and Post Test, with Key. 89 

E. Participants Institute Evaluation Form 99 



85 

o 

ERLC 

hffliflaffBEaaaa 



78 



APPENDIX A 



Selected References Recommended 
for those Interested in Descriptive Research 



J ournals 

American Jou rnal of Art Therapy . (Formerly, The Bulletin of Art 
Therapy). (4 times a year). 6010 Broad Branch Rd. , N.W., Washing- 
ton, D. C. 20015. (Subscription $5. 00 a year). 

J ournal of Aesthetic Education. (‘1 times a year). University of Illinois 
Press, 1002 W. Green St. , Urbana, Illinois 61801. ($7. 50 a year). 

Journ al of Aest hetic s and Art Criticism . Published quarterly by the 
American Society for Aesthetics at Wayne State University: College of 

Liberal Arts and University Press, and The Cleveland Museum of Art, 
Mt. Royal and Guilford Avenues, Baltimore, Maryland 21202. 

Journal of Creative Behavior. (4 times a year). Creative Education 
Foundation, State University College at Buffalo, 1 300 Elmwood Ave. , 
Buffalo, N. Y. 14222. ($8.00 a year). 

Review of Educat ional Researc h. {Now published 5 times a year). 
American Research Education, 1126 l6th St. , N.W., Washington, D. C. 
20036, This journal cyclically (3 year cycles) reviews fields and re- 
search topics. Last reference on Art Education: Vol. XXXVII, No. 2, 

1967. 

Studies in Art Educa tion. A journal of issues and Research in Art 
Education. (3 times a year - Fall, Winter Spring). 1201 L 6th St. , 

N. W . , Washington, D.C. 20036. ($5.00 a year, or as part of the 

comprehens i ve membership, along with one gift new publication, a 
$20. 00 membership). 



Books and Reviews 

AS em i n a r in Art Ed u c atio n (or Re s ea rch and Cur ? i culum Develo p ment . 
E. L. Mauil, ed. C.R.P, No V-002. University Park: The Pennsyl- 

vania State University; 1966. 
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Arte and Humanities Educational Rese a rch Projects . U, S, Depart- 
ment of Health, Education, and Welfare, Office of Education, Bureau 
of Research, Arts and Humanities Program, Washington, D, C, , 20202; 

July 15, 1969. 

Bloom, et. al . Taxonomy of Education al Ob ject ives. Handbook I: Cog- 

nitive Domain (1956), Handbook II: Affective Domain (1964). 

Crenbach, L, J, Essentials of Psychological Testing . Harper; 3rd, ed, 
1969. 

Dissertation Abstracts . Ann Arbor, Michigan: Unive rsity Microfilms, 

(Title varies: 1938-1951, Microfilm Abstracts ; 1952 to present, Dis- 

sertation Abstracts ). 



Encyclopedia of Ed. Res . Robert L. Ebel, ed. New York: MacMillan; 
1969. Also see: Encyclopedia of Ed. Res. Chester W. Harris, ed. ; 
I960, —— — — 

Gage, N. L. Handbook of Reseaich on T eaching . Chicago: Rand 

McNally; 1963. (Now being revised,) 

Games, P. A. and Klaire, George R, Elementary Statistics. McGraw- 
Hill; 1967. 

Harris, Dale B, Children^ Drawi n gs as Measures of Intellectual 
Maturit y, Ha r court. Brace, World; 1963. 

Kerlinger, Fred. Foundations of Behavioral Re search. Holt* Rinehart 
and Winston; 1964. 

Lanier, Vincent. Doctoral Research in Art Ed a cation. University of 
Southern California; 1962. (i\pdated with later , . pplements). 
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APPENDIX B 



P RE CONFERENCE PROGRAM* 

Research Training Institutes for 
Descriptive Research 



DAY ONE 



8:00 - 9:00 AM 
9:00 AM 



10:30 - 12 Noon 



12 Noon - 1:30 PM 
1:30 - 3:00 PM 



3:00 - 5:50 PM 



Registration 

FIRST SESSION - LARGE GROUP 
Welcome 

Introduction of Staff 
Program Orientation 
P ~e -T e st 



SECOND SESSION - LARGE GROUP 

Lecture and Discussion (parts of 1 L 11) 
Drawing problem for participants 

Lunch 

THIRD SESSION - SMALL GROUPS 

Using Drawings: Identify criteria, define 
critera, develop simple scales of vari- 
ables . 

FOURTH SESSION - LARGE GROUPS 
Lecture and discussion (parts 1 L 1 1) 

Small Group reports on work 
Quo stions 

Evaluator's comments on events of Day One 



* This is a resume of the programs with brief descriptions of the con- 
tents of each session. The information in the parentheses refers to 
the parts of this publication that were used in the lectures. 
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DAY TWO 



9:00 - 9:45 AM 


FIFTH SESSION - LARGE GROUP 
Lecture and Discussion (part of 1^) 


9:45 - ) 1:00 AM 


SIXTH SESSION - SMALL GROUPS 

Work on problem selection and refining it 


11:00 - 12:30 


SEVENTH SESSION - LARGE GROUP 

Lecture and. Discussion (Remainder of IV 
and part V) 


12:30 - 2:00 PM 


Lunch 


2:00 - 4:00 PM 


EIGHTH SESSION - SMALL GROUPS 

Continue refining researchable problem 
and begin development of measuring 
instruments where appropriate. 


4:00 - 5:30 PM 


NINTH SESSION - LARGE GROUP 

Lecture and Discussion (remainder of V) 
Discussion of problems encountered in 
small groups 
Questions 

Evaluato 1 s comments on Day Two 



DAY THREE 



9:00 - 10:30 AM 


TENTH SESSION - LARGE GROUP 
Lecture and Discussion (111) 


10:00 - \Z Noon 


ELEVENTH SESSION - LARGE GROUP 
Questions and discussion of problems 


12:00 - 1:30 PM 


Lunch 


1:30 - 3:00 PM 


TWELFTH c ESSION - SMALL GROUPS 

Final work session. Preparing final form 
of research problem, design, and measur- 
ing instruments. 


3:00 - 5:00 PM 


TH T RTEENTH SESSION - LARGE GROUP 
Post -Test 

Reports on work from small groups 
Final questions and discussion 
Evaluator's comments 
Farewell 



APPENCIX C 



EVALUATION REPORT 
Prepared by William Rabinowitz 
1970 NAEA 

Preconference Educational Research Training Program 
In Art Education 
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1 , Evaluation Planning 

The L970 NAEA Preconference Educational Research Training Program 
consisted of our sessions as follows: 

March 7-9, 1 97 0 / She raton Hotel/ Philadelphia, Pennsylvania 
(NAEA Eastern Regional Conference) 

MarOi 19-21, 1970/ Sheraton-Schroeder Elotel/ Milwaukee, 
Wisconsin (NAEA Western Regional Conference) 

April 12-14, 1970/ John Marshall Hotel/ Richmond, Virginia 
(NAEA Southeastern Regional Conference) 

April 23-25, 1970/ Hilton Hotel/ Portland, Oregon 
(NAEA Pacific Regional Conference) 

Each of these sessions preceded the regularly scheduled regional meet- 
ing of the NAEA, 

A decision concerning the U, S, Office of Education's willingness to 
sponsor the program was communicated shortly before the Philadelphia 
session was scheduled to begin. There was therefore little time 
available in which to plan a systematic and thorough evaluation. Never- 
theless, it was possible for members of the planning committee 
(Drs, G. Kensler, D. B. Har:is, E. Matlil, S. Madeja, and J. Mahlmann) 
to meet on February 27, 1970, and to plan details of the program includ- 
ing aspects of the evaluation. 

It was agreed that the formal evaluation would consist of a multiple choice 
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objective test (to measure student comprehension of basic principle 0 of 
descriptive research) and a questionnaire (to measure student reactions 
to various facets of the organization, presentation, and content of the 
program). Preparation of these instruments was to be the responsibil- 
ity of the Program Evaluator, Dr. W. Rabinowitz. 

In preparing the objective test, the Evaluator worked closely with the 
Research Specialist, Dr. D. B. Harris. Since Dr Harris had primary 
responsibility lor conducting the large-group lecture sessions during 
the program, it was important that the test cover the same material lie 
planned to present. Although the ' 1 fit’ 1 between Dr. Harris' Instruction 
and the objective test was reasonably close, it was not possible in the 
limited time available to construct a test that perfectly reflected the 
concepts presented in the program. 

In preparing the questionnaire, the Evaluator worked closely with the 
Project Director, Dr. G. Kensler, and Dr. J. Mahlmann of NAEA. 

Since Drs. Kensler and Mahlmann were primarily responsible lor the 
planning and administration of the program, it was important that the 
questionnaire cover aspects of the program in which they were especially 
interested. 

Copies of the objective lest and questionnaire used as evaluation instru- 
ments are appended to this report. 

11, Evaluation Activities 

Each preconference program consisted of three days of instruction and 
discussion. During the first meeting of the participants, immediately 
following some brief words of welcome and introduction, the objective 
test was administered. All copies of the test and answer sheets were 
collected and scored at this time. The objective test was administered 
once again daring the afternoon of the final day of the conference. An- 
swer sheets were collected, but participants, at this time, were per- 
mitted to retain copies of the tests and were given an answer key. The 
questionnaires were also administered during the afternoon of the final 
day of the conference. 

In addition to administering the test and questionnaire, the Evaluator 
functioned as an observer of the lecture and small-group sessions. 

During ’’breaks" the Evaluator discussed aspects of the program with 
staff and par i Icipants . At the end of each day, during a brief session, 
the Evaluator reported informally to staff and participants on a variety 
of matters of concern. These feedback sessions included, for example, 
di: cussions of the test results, areas of special concern among partici- 
pants, similarities and differences in the activities of small groups, 
etc. 
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111. Evaluation Results 



The evaluation results will be discussed separately for the objective 
test and the que stionnai re . 

Objective Test 



The objective test was administered as both a pretest and post test, 
making it possible to assess directly the gain of each participant. The 
results at the four preconfe rence sessions did not differ significantly 
from one another, and they have therefore been combined. Based on a 
total of 89 participants, for whom both prc- and pest test data were 
available, the following results were obtained: 





Mean 


Standard Deviation 


P retest 


22, 9 


4. 2 


Post Test 


27. 3 


3 . 6 



The mean gain of +4.4 points is significant at the .01 level (t - 10. 82, 
d. f. = 88). 

Although the gain was statistically significant, it was clearly not very 
impressive. Almost all of the students showed smiie gain in score (a 
few gained more than 10 points), but many complained that the post 
test did not adequately reflect what they had learned during the precon' 
fc rence sessions. In some instances, this was attributed to the imper- 
fect correspondence between the test and the content of the instruction*, 
in other instances, it was attributed to difficulties with the multiple - 
choice format. It seems highly likely that, although the test provided 
evidence of significant, relevant learning, it did not reveal the (ull 
extent to which this learning occurred during the program. 

Que sti onna ire 

Completed que stionnaire s were available from 87 participants. As in 
the case of the objective test, the results ai the four prc come rence 
sessions did not differ significantly from one another, and they have 
therefore been combined. The responses of the participants are sum- 
marized below for the e ght que s tionnai re items in which a direct 
tabulation of response is possible. 

1. How would you e aluate the organization of the program? 
Percent 

Excellent organization in clear and meaningful 

sequence 
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54 Gene rally well -organ! zed 

11 Adequately organizeJ but could have been better 
0 Inadequately organized, too much confusion 

0 Very disorganized, no apparent structure 

2, How realistic and attainable were the objectives of the pro- 
gram ? 

Pe rcent 

17 Very realistic and easily attainable 
49 Most of the objectives were clearly attainable 
22 Many of the objectives were attainable, but some 
were not 

1 Most of the object' ves were not attainable 

_l The objectives of the program were clearly unreal- 

istic 

10 No response 



3. To what extent did the program meet your prior expectations? 
Pe rcent 

2 1 The program exceeded my expectations 
53 My expectations were generally well-met 
1 3 Tlu program was O, K, , but not all my expectations 
were met 

10 1 expected more from the program than I obtained 

0 The program completely failed to meet my expect- 
a turns 

3 No response 



4* To what extent did the program deal with questions and 
problems that are applicable to your needs? 

Pe rcent 

34 _ The program was exceptionally well -related to my 
need s 

34 The program was generally well -related to my needs 
24 The program was adequate in terms of my needs 

The program was only slightly related to my needs 

0 1 he program was completely unrelated to my needs 

1 Xn response 



5. 



Was the length ol the program o ppropriat c ? 

Pe rcent 

5 Considering what was covered, the program was too 
long 

63 The length of the program was just right 
30 Considering what was covered, the program was too 
s h o r t 

2 No response 
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6. Was the difficulty level of the program appropriate? 

Percent 

6 Considering my background, the program was too 
difficult 

74 The difficulty level of the program was just right 
l(> Considering by background, the program was too 

ea sy 

4 No response 

7. In general, how would you evaluate the quality of the instruc- 
tion ? 

Pe rcent 

55 Outstanding 
37 Good 

6 Satisfactory 

1 Inadequate 

0 Poor 

1 No Response 

8. Would you recommend that the NAEA continue to sponsor 
research programs in the future? 

Percent 

99 Yes 

1 No 

The questionnaire responses clearly indicate that, in general, the 
participants were quite positive about their experience in the program. 
More than 90 percent of the participants, for example, considered Jie 
quality of the instruction ’’excellent' r or ’’good”, and almost 70 percent 
felt that the program was well -related to their needs. The progran 
was generally considered well -or ganized and appropr. ite in length and 
difficulty level. Almost two-thirds of the participants described the 
objectives of the program as clearly or easily attainable, and almost 
th ree -fou rths felt that their expectations were well -met or exceeded by 
the program. Ninety -nine percent recommended that the NAEA con- 
tinue to sponsor such programs in the future. 

The questionnaire encouraged respondents to make any relevant com- 
ments concerning the program when they felt so inclined. An examin- 
ation of these comments is instructive. For example, a number ol 
constructive criticisms were offered. Several participants felt the 
need for a stronger orientation concerning the objectives and overall 
purpose of the program. Some suggested that the objectives should 
have been ’’spelled out” durmg the first session of the program: others 
felt that a more detailed and explicit description of the program should 
ha* r e been distributed in advance. The small group sessions were, in 
general, not evaluated as highly as the large group Lectures. There 
was a feeling, expressed by somt participants, that the purpose of the 



small group sessions was unclear. Others were disturbed by what 
they considered a lack of relationship between the lectures and the 
small group activities. As one participant ^ut it, 1! We should have had. 
more structure and direction from the leaders in the small groups. It 
was hard to relate what we were doing to the concepts discussed in the 
lectures . M 

Although most of the comments were implicitly or explicitly critical, 
they did not significantly modify the general picture of satisfaction with 
the program. Most of the participants would probably have agreed with 
one who said, n M.y expectations were vague when I came, and I didn T t 
really krow what to expect. I've not been disappointed, however. This 
has been much better than previous seminars, w 
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APPENDIX D 



FORMAL EVALUATION TEST WITH KEY 
1970 NAEA 

Prcconfcronce Educational Research Training Program 
In Art Education 



Directions: Record your answer on the separate answer sheet. Do not 

quess wildly. 

1. The first step in any research study is 

a. selecting the statistical procedures that will be used. 

b. choosing a good design. 

c. selecting the persons who will be studied, 

d. specifying the question(s) to be answered. 

2. In oriicr for a question or problem to provide a useful basis for re- 
search, it must be 

a. free from any implied value judgments. 

b. stated in quantitative, rather than qualitative, form, 

c. answerable through some empirical procedure, 

d. based on an explicit educational theory. 

3. Which of the following is most likely to suit from formulating the 
objectives of instruction in behaviora’ ms? 

a. Students will Learn in a more icient manner, 

b. Measures of student achieve/ .unt based on instructional 
objectives will be developed. 

c. Subjective factors in the evaluation of the student will be 
eliminated.. 

d. The student's behavior will be the basis tor establishing the 
goals of instruction, 
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4. Characteristics of persons or things which can assume different 
values are called 

a, distributions, 

b, observations, 

c. data. 

d. variables. 

5. An operational definition of a variable is one expressed in terms of 

a. mathematical symbols. 

b. action-related postulates. 

c. antecedent-consequent relationships. 

d. observable events. 

6. Which of the following represents the crudest level of measurement? 

a. J >e cm draw. 

b, Joe can draw very well. 

c. Joe can draw better than Bill, but not as well as Mary, 

d, Joe can draw better than he can paint, 

7. No matter how a research study is conducted, the data ultimately 
consist of 

a, statistical indicators. 

b, rr liable measurement s. 

c, human observations. 

d, unwarranted inferences, 

8. A product scale would be most appropriate to evaluate performance 
in 

a, drawing, 

b, arithmetic. 

c, spelling, 

d, reading, 

9 . Research studies which attempt to determini the relative incidence, 
distribution, and inter-relations among various sociological and 
educational variables in populations by selecting, and studying 
samples are called. 




a. surveys. 

b. field experiments. 

c. normative studies. 

d. cros s sectional investigations. 
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10. Observations planned in advance are preferable to ratings based 
on past experiences with a person because planned observations 
will 

a. cover a wider range of types of behavior. 

b. include a longer time period. 

c. be more economical to obtain. 

d. be more objective and closer to actual behavior. 

11. One important virtue of direct observation is that the method 

a. is economical and efficient. 

b. digs into the inner motives of the individual. 

c. can be applied in natural real-life situations. 

d. yields a record of behavior that is directly meaningful. 

12. Direct observation would be most useful as a research method if 
one were trying to answer the question, 

a. How are programs in art staffed in relatively more and less 
affluc it schools? 

b. Is artistic ability related to creativity? 

c. Can an intensive program in art help children, to read anc 
write better? 

d. Are there any differences between lower - and middle -class 
chile- run in their use of artistic media? 

13. Direct observation would be most likely to be used in a study of 

a. attitudes of parents toward the school. 

b. teachers' classroom behavior. 

c. children's concepts of life and death. 

d. learning and forgetting under varied conditions of practice. 

14. With what sorts of individuals has the method of direct observation 
proven particularly useful? 

a. Those who are culturally disadvantaged. 

b. Young children, 

c. Individuals from a different culture, 
c. Sophist. cated adults. 

15. Which of the following is most characteristic of skilled as oppose*, 
to un skilled ob -servers in research? 

a. Making generalized descriptions or evaluations rather than 
noting specific incidents. 
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b. Interpreting behavior on the basis of limited data rather 
than waiting for confirmation by subsequent incidents* 

c f Recording the child’s behavior rathe::* than the personal 
reaction of the observer. 

d. Recording primarily dramatic, unusual, or negative inci- 
dents. 

16. Sampling of behavior by observational methods can be considered 
as consisting of two aspects, 

a. person sampling and event sampling. 

b. person sampling and time sampling. 

c. event sampling and time sampling. 

d. criterion sampling and person sampling. 

17. From the reliability coefficient of a tost one can judge 

a. how consistently the pupil will maintain his position in the 
group if an equivalent test is given. 

b. hosV many points the pupil is likely to change if an equival- 
ent test is given, 

c. whether the test is measuring what it is supposed to measure, 

d. whether the test is related to other significant factors in the 
individual. 

18. n Halo ,r effect refers to the 

a, influence of one rater upon another. 

b, tendency to rate a person higher when you know him better, 

c. spread of general impression of a person to the rating of 
many special characteristics. 

d. tendency to make ratings too high. 

19. Generosity ^ *or is illustrated by the fact that 

a. few people are ever rated below average, 

L>, higher ratings are given to close acquaintances. 

c. a person who is rated high cm one trait is usually latcd high 
on other traits also, 

d, one leans over backwards not to be too hard in rating people 
one doesn't like, 

20. Annual achievement tests which are pirts of the school testing pro- 
gram, illness records, daily assignments and other routine char- 
acteristics arid activities of the school are considered to be 




a. objective measures. 
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b, ronreactive measures, 

c, reliable measure. 

d, standardized measures. 

21. As part of a research study a group of 50 teachers were to be 
rated on the clarity with which they presented concepts of their 
pupils. Each teacher was to be rated by an observer after a visit 
of an hour. To make the ratings more reliable, it would probably 
be most important to 

a. extend the visit to two hours. 

b. make several different ratings. 

c. visit each teacher several limes. 

d. increase the number of teachers 

22 . When subjects perform well merely because they are being observed 
(and not necessarily because of any effect of treatment), this is con- 
sidered to be an example of the 

a. Hawthorne effect 

b. novelty effect. 

c. halo effect. 

d. observer effect. 

2h In a research study an observer may select categories of events to 



be obse 


rved on the basis of 


a. 


theoretical grounds. 


b. 


practical considei ations 


c 


empirical evidence. 


d. 


All of the above. 



24. For educational purposes, classification schemes are 

a. of little use, because they allow for neither prediction nor 
control of behavior. 

b. of limited use, if they are arranged on a meaningful 
hierarchical oasis. 

c. potentially very useful, depending on the quality and preci- 
sion with which categories are defined. 

d. extremely useful, given the inadequate development of 
educational measurement. 

2 5. A questionnai re and an interview arc alike in that 

a. the situation is standardized and uniform for eacn respon- 
dent. 
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b. the basic information is supplied by the subject in response 
to questions. 

c. the result is a score or scores arrived at in the same wa^ 
for each person, 

d. the two techniques yield objective information. 

26. Separating a group of possible subjects into categories (e,g #> males 
and females) before selecting subjects (by use of a table of random 
numbers) to receive a particular treatment is known as drawing a 

a< stratified random population, 

b. simple random sample. 

c. unbiased random sample, 

d. stratified random sample. 



27. It is important to follow up those persons in a survey-sample who 
have failed to return questionnaires mailed to them? (Assume that 
the direct-mail method is appropriate,) 



a. No, because a certain loss must be expected. 

b. No, because such delayed returns would be of doubtful value. 

c. Yes, because the sample at hand may be biased, 

d. Yes, because the size of sample should be as large as pos- 
sible* 



28. Which of the following is usually the most serious limitation in the 
use of the mailed questionnaire in descriptive research? 

a. Defining a population, 

b. Selecting a sample. 

c. Obtaining representative responses. 

d. Analyzing the data. 



29. As compared with laboratory experiments, studies in school set- 
tings generally have 



a. less control over relevant variables. 

b. m ore omtrol ovk r relevant variables. 

c. abuut t.qual control over relevant variables. 

cl. it is impossible to make u comparison in this regard. 



30. A procedure in which the researcher observes the child and re- 
cords his responses is preferably to one in which the child marks 
an answer sheet in that it 




a. is more objective and accurate, 

b. is more economical of time and money, 
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c, permits observation of the process of reaching the answer. 

d. corresponds more closely to actual school conditions. 

31. In the case of some traits or achievements, we arc limited to 
rather rough subjective evaluations. Wc should 

a. avoid the evaluation of such traits, because our techniques 
are subjective. 

b. center our attention on the traits which we can measure 
objectively. 

c, be content with the subjective procedures, since they are 
all we have. 

d, use the subjective procedures with caution, while trying to 
develop improved techniques, 

32. One of the serious limitations of the information that is often 
obtained from systematic behavior observations is that 

a. it usually depends too much on interpretation of what the 
person is doing. 

b. the specific behaviors may be so isolated and external that 
it is hard to know what they signify. 

c. it is limited to unreal and artificial situations, 

d. it is dependent upon the cooperation of the subjects, 

33. If we are trying to use ratings to provide an evaluation of the effec- 
tiveness of teachers, we may replace 3 or *1 broad trait names by a 
list of 30 or 40 specific behaviors. This may have various results, 
such as 

1, greater uniformity of meaning from one rater to an- 
othe l*. 

11. less relationship of the ratings to actual observations 
of the ratees 1 behavior. 

111. more difficulty in using the ratings to remedy indi- 
vidual strengths and weaknesses. 

Which of the above arc likely to occur? 

a. 1 only 

b. 1 and 1 1 only 

c . 1 and 1 1 1 only 

d . II and 1 1 1 only 
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34. Which of the following is equivalent to increasing the length of a 
te st ? 

a. increasing the number of pupils tested. 

b. increasing the time allowed for giving the test. 

c. increasing the number of raters rating pupils. 

d. increasing the range of grades tested. 



35. A pupil's interest in entering an experimental art program could 



probably be determined most adequately bv a 



a. self-report inventory. 

b. situational test. 

c. projective method. 

d. teacher's rating. 

36. Interviews and que svionnaire s as data collection methods 



a. are more effective than observational techniques, 
b t reveal only information the subject is willing to report, 

c. cannot be considered to have validity. 

d. provide no information about past behavior. 



37. A research program is being sot up to develop tests to use in the 
selection of students for a new art program. It is probable that 
the greatest difficulty would arise in 

a, selecting promising tests to try out. 

b, getting the cooperation of a group of students, 

c. working out statistical procedures for determining test 
validitie s . 

d. obtaining satisfactory measures of success in the program, 

38, When a researcher states that a result is significant, he means 

a. the effect is a practically important one. 

b. the scores are highly correlated. 

c» the iu.suU is unlikely to be a chance- occurrence, 
d. tile sample values arc ciiifcruul from one another. 



39. Tlie statistical metheds to be used in a study should be considered 

a. before the hypotheses have been formulated. 

b. as the study is being designed. 

c. after the data have been collected, 

d. after the data have been tabulated. 
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40 . 



When a research er prepares a well-designed set of charts shov/ing 
the results of a study, the education?.! significance of the results 
should be 

a, immediately apparent from the charts, 

b, supplied in accompany ing captions or text. 

c, left to the reader to work out for himself, 

d, a matter for each individual's own education philosophy. 
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APPENDIX E 



PARTICIPANTS EVALUATION FORM 
\ 97 0 NAEA 

Preconfe reace Educational Research Training Program 
In Art Fjducation 



Please complete this form and return it to the Evaluator, William 
Rabinowitz, at the end of the program. 



1, How would you evaluate iho organization of the program? 

Excellent organization in clear and mcaningfuL sequence 
Generally well -organized 

Adequately organized, but could have been better 
Inadequately organized, too much contusion 

Very disorganized, no apparent structure 

Comments: 




2. How realistic and attainable were the objectives of the program? 
Very realistic and easily attainable 
Most of the objectives were clearly attainable 
^ Many of the objectives were attainable, but some were nut 
__ __ Most of the objectives were not attainable 

The objectives of the program were clearly unrealistic 
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Comments: 



3, To what extent did the program meet your prior expectations? 

The program exceeded my expectations 

My expectations were generally well -met 

The program was O, K, , but not all my expectations were 

met 

I expected more from the program than I obtained 

__ _ The prog rani completely failed to meet my expectations 

C om ni e nt s : 



4, To what e x . v . : t did the program deal with questions and problems 
that e r<> applicable to your needs? 

_ The program was exceptionally well-related to my needs 

_ 'Luc program was generally well-related to my needs 

The program was adequate in terms of my needs 

1 he program was only slightly related to my needs 

The program was completely unrelated to my needs 

Comment : 
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5. Was the length of the program appropriate? 



Considering what was covered, the program was too long 
The length of the program was just right 

Considering what was covered, the program was too short 

Comments: 



6, Was the difficulty level of the program appropriate? 

Considering my background, the program was too difficult 

The difficulty level of the program was just right 

Considering my background, the program was too easy 

Comments: 



7, In general, how would you evaluate the quality of the instruction? 
Outstanding 
Good 

Sat isfa clo ry 
_ Inadequate 
Poor 

Comments: 
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8. Please indicate in the space below one or two ways in which you 
plan to use ideas or skills acquired during the program. 



9. Would you recommend that the NAEA continue to sponsor research 
programs in the future? 

Yes _____ No 

ID. Please indicate in the space below your suggestions for topics or 
problem areas for future programs. You may also indicate a ny 
oilier general suggestions tor conducting future pro^ rams. 



Signature: 



(Note: This :s optional. 
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